Analysis of data obtained from microarrays

ABSTRACT

Disclosed are methods and software for biological data analysis. Specifically, provided are methods, computer programs and systems for analyzing data in the form of various intensity measurements obtained from an oligonucleotides microarray experiment. Such data may be microarray data obtained from an experiment conducted to determine copy number of a human genetic sample. The data are corrected by application of one or more covariate adjusters which may be applied simultaneously and which may be selected by a user. Further, the present application provides methods of filtering image data and signal restoration of image data using log2 ratio data.

RELATED APPLICATIONS

The present application claims priority from U.S. Provisional PatentApplication No. 61/564,754, filed Nov. 29, 2011 which is herebyincorporated herein in its entirety for all purposes.

FIELD OF THE INVENTION

The presently disclosed methods and software are related tobioinformatics and data analysis including methods, computer programsand systems for analyzing intensity measurements obtained from anoligonucleotide microarray experiments. Included are methods oftransforming and displaying intensity data read from microarrays.Transformations include the application of various covariate adjustors.Further included are methods of adjusting and refining displayparameters of intensity data. Intensity data includes, for instance,fluorescence intensity measured from labeled genetic material hybridizedto a microarray using a scanner.

BACKGROUND OF THE INVENTION

Single nucleotide polymorphism (SNP) and genetic copy number (CN) havebeen used extensively for genetic analysis. Fast and reliablehybridization-based SNP/CN assays have been developed. (See, Wang etal., Science, 280:1077-1082, 1998; Gingeras, et al., Genome Research,8:435-448, 1998; Halushka, et al., Nature Genetics, 22:239-247, 1999;Korbel t al., “Systematic prediction and validation of breakpointsassociated with copy-number variants in the human genome,” PNAS USA,104(24):10110-10115, 2007; and Nigel P. Carter, “Methods and strategiesfor analyzing copy number variation using DNA microarrays,” NatureGenetics, 30:S16-S21, 2007, incorporated herein by reference in theirentireties). Computer-implemented methods for discovering polymorphismand determining genotypes are disclosed in, for example, U.S. Pat. No.5,858,659 (incorporated herein by reference in its entirety for allpurposes). However, there is still need for additional methods fordetermining genotypes and displaying the large amount of geneticinformation obtained from such experiments in a user-friendlyinteractive computer application.

Data can be statistically manipulated to eliminate independent variablesby use of covariate adjusters. Sample data obtained from microarrayexperiments is in the form of intensity values. Intensity valuescorrespond to the hybridization of labeled genetic material to a probemounted on a microarray. Such intensity values can have many intrinsicindependent variables that are unrelated to the variable being studied,and which can confound the result, i.e. mask the result to yieldunpredictable results. Strides have been made in the past to removethese independent variables so that genetic sample analyses onmicroarrays may be more consistent and of a higher quality, i.e. morereliable and determinative when studying disease. However, manyvariables still plague data and sample analyses. Likewise, display ofsuch intensity measurements in genomic microarray studies can beinfluenced by many independent variables. Much can be done to eliminatethese variables to reveal underlying patterns and signals impacting datainterpretation. Various image data filters and data manipulationsstrategies may be employed to remove these independent variables toagain improve consistency and quality of data presentation and analysis.The discovery of new ways of increasing the quality of these data and ofthe scanning, measuring and displaying of these intensities are in direneed to keep pace with the rapid advancement of the application ofdiagnostic utilities associated with microarray-based genetic tests.

BRIEF SUMMARY OF THE INVENTION

Methods of analyzing and manipulating intensity value data aredisclosed. The methods include normalization of intensity value data toremove independent variables by use of various covariate adjusters.Other methods, including methods of filtering intensity value data whichis visually displayed, are also disclosed as well as methods of geneticcopy number signal restoration. Various algorithms and computer programsare disclosed for carrying out the methods.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further features will be more clearly appreciated from thefollowing detailed description when taken in conjunction with theaccompanying drawings. In functional block diagrams, rectanglesgenerally indicate functional elements and parallelograms generallyindicate data. In method flow charts, rectangles generally indicatemethod steps and diamond shapes generally indicate decision elements.All of these conventions, however, are intended to be typical orillustrative, rather than limiting.

FIG. 1 provides an exemplary flow chart for statistical analysis ofmicroarray data. Steps related to probes used for copy numberdetermination run down the left side of the flow chart and steps relatedto SNP detection are along the right side of the flow chart.

FIG. 2 is a boxplot, displaying various value bins of intensity, withthe center of each box summarizing the relationship between signal andcovariate. The x-axis plots fragment length in base pairs (bp) and the yaxis plots signal intensity in relative intensity units.

FIG. 3 is a plot summarizing the signal intensity measurement using theoverall median. The x-axis plots fragment length in base pairs (bp) andthe y axis plots signal intensity in relative intensity units. The lineacross the middle represents the target overall median intensity.

FIG. 4 is a plot showing how the signal intensity measurement for eachvalue within each bin is manipulated to fall at the level of the overallmedian. The x-axis plots fragment length in base pairs (bp) and the yaxis plots signal intensity in relative intensity units. The line acrossthe middle represents the target overall median intensity.

FIG. 5 is a plot showing the individual adjustment factors for each binof signal intensity value. The x-axis plots fragment length in basepairs (bp) and the y axis plots the relative adjustment made to each binin relative units.

FIG. 6 visually depicts the post-adjustment result of signal intensitiesscaled using the computed scale factor of FIG. 5 on the signalintensities of FIG. 2. The x-axis plots fragment length in base pairs(bp) and the y axis plots signal intensity in relative intensity units.

FIG. 7 is a flow chart depicting an exemplary procedure for applying thecovariates and algorithms of the present methods and systems.

FIG. 8 depicts a pre-adjustment (before application of covariate) plotof log₂ ratio for binned signal intensities, binned by percent GCcontent. The x-axis plots GC content in percentages (% peroligonucleotide fragment) and the y axis plots signal intensity inrelative intensity units. The line across the middle represents thetarget overall median intensity.

FIG. 9 shows the bins summarized based on their corresponding median log2 ratios. The x-axis plots GC content in percentages (% peroligonucleotide fragment) and the y axis plots signal intensity inrelative intensity units. The line across the middle represents thetarget overall median intensity.

FIG. 10 shows how the differences for each bin are eliminated byshifting each median to the value of zero. The x-axis plots GC contentin percentages (% per oligonucleotide fragment) and the y axis plotssignal intensity in relative intensity units. The line across the middlerepresents the target overall median intensity.

FIG. 11 provides the finalized boxplot of log 2 ratio values adjusted bysubtraction of the median value that corresponds to each covariate binfor each bin. The x-axis plots GC content in percentages (% peroligonucleotide fragment) and the y axis plots signal intensity inrelative intensity units. The line across the middle represents thetarget overall median intensity.

FIG. 12 is an exemplary log 2 ratio adjustment flow chart.

FIG. 13A illustrates a covariate that is turned into a discrete variableby means of binning. FIGS. 13B-C illustrate two exemplary alternativesfor binning a covariate variable using either equal counts or equalspacing to determine which data is allotted to which bins and theeffects of each method on the spread of the bins across the data.

FIG. 14 is an exemplary flow chart which could be used for buildingreference copy number values in single sample assay (SSA) mode. Theflowchart shows where algorithms of the present methods and systems maybe employed in the determination of a reference copy number in astandard microarray experiment. The input is standard CEL file data andthe output is the final determination of copy number reference values.

FIG. 15 is an exemplary single sample analysis flow chart showing wherealgorithms of the present methods and systems may be employed in thesample processing of a microarray experiment. Copy number specificmanipulations are provided in the lefthand box while SNP specific dataprocessing steps are provided in the middle box. Outputs in therighthand box include a CYCHP file and a report file, commonly employedin contemporary software applications used to sort, analyze andvisualize such microarray experimental data.

FIG. 16 represents discretization of three different generic covariates,with each marker being represented as a point in three-dimensionalspace.

FIG. 17 provides the first step in a method embodiment in which just twogeneric covariate markers are depicted for analysis.

FIG. 18 provide subsequent steps to the two dimensional depiction of themarkers for two generic covariates, in which the intercept of onecovariate with another is estimated and subtracted from the othergeneric covariate and the axes then rotated such that the regressionline becomes the new horizontal axis, yielding a new coordinate system.

FIGS. 19A-D are visual depictions of the binning or partitioning of thetransformed space of the covariate discretization process. FIG. 19Arepresents the initial transformed space. The space is then divided intoequally populated bins by using linear separators, as depicted in FIG.19B. Each of these bins is then further divided, as in FIG. 19C. Afurther round of partitioning occurs, as shown in FIG. 19D.

FIG. 20 depicts the 625 distinct bins in the exemplary fragment GC andlocal GC space.

FIGS. 21A-D provide exemplary values of a fragment length covariate,broken down by CN or SNP marker and also by chromosome. FIG. 21Adisplays the relationship between fragment length and copy numbermarkers and FIG. 21B displays the relationship between fragment lengthand SNP markers. FIGS. 21C-D display the relationship between fragmentlength and chromosome number.

FIGS. 22A-B provide exemplary values of a local GC content, broken downby CN or SNP marker. FIG. 22A displays copy number markers and FIG. 22Bdisplays SNP markers.

FIGS. 23A-B provide exemplary values of a fragment length covariate,broken down by CN or SNP. FIG. 23A displays copy number markers and FIG.23B displays SNP markers.

FIGS. 24A-D provide exemplary values of a probe GC, broken down by CN orSNP marker and also by chromosome. FIG. 24A displays the relationshipbetween probe GC content and copy number markers and FIG. 24B displaysthe relationship between probe GC content and SNP markers. FIG. 24Cdisplays the relationship between chromosome number and copy numbermarkers and FIG. 24D displays the relationship between chromosome numberand SNP markers.

FIGS. 25A-B provide exemplary values of an adapter type GC, broken downby CN or SNP marker. FIG. 25A displays copy number markers and FIG. 25Bdisplays SNP markers.

FIGS. 26A-B show pair-wise comparison of fragment GC and fragment lengthcovariate adjusters. FIG. 26A displays copy number markers and FIG. 26Bdisplays SNP markers.

FIGS. 27A-B show pair-wise comparison of fragment GC and local GCcovariate adjusters. FIG. 27A displays copy number markers and FIG. 27Bdisplays SNP markers.

FIGS. 28A-B show pair-wise comparison of fragment GC and probe GCcovariate adjusters. FIG. 28A displays copy number markers and FIG. 28Bdisplays SNP markers.

FIGS. 29A-B show pair-wise fragment length and probe GC covariateadjusters. FIG. 29A displays copy number markers and FIG. 29B displaysSNP markers.

FIG. 30 depicts an exemplary flowchart for use of the covariateadjusters in the methods of the present invention for building aprototype reference.

FIG. 31 depicts an exemplary flowchart for single sample processingemploying the presently disclosed covariate adjusters.

DETAILED DESCRIPTION I. General Description

Reference will now be made in detail to exemplary embodiments of theinvention. While the invention will be described in conjunction with theexemplary embodiments, it will be understood that they are not intendedto limit the invention to these embodiments. On the contrary, theinvention is intended to encompass alternatives, modifications andequivalents, which may be included within the spirit and scope of theinvention.

The invention relates to diverse fields impacted by the nature ofmolecular interaction, including chemistry, biology, medicine anddiagnostics. Methods disclosed herein are advantageous in variousscientific, medical and diagnostic fields, such as those in whichgenetic information is required quickly, as in clinical diagnosticlaboratories or in large-scale undertakings such as the Human GenomeProject.

The invention described herein has many embodiments and relies on manypatents, applications and other references for details known to those ofthe art. Therefore, when a patent, application, or other reference iscited or repeated below, it should be understood that the entiredisclosure of the document cited is incorporated by reference in itsentirety for all purposes as well as for the proposition that isrecited. All documents, e.g., publications and patent applications,cited in this disclosure, including the foregoing, are incorporatedherein by reference in their entireties for all purposes to the sameextent as if each of the individual documents were specifically andindividually indicated to be so incorporated herein by reference in itsentirety.

As used in this application, the singular form “a,” “an,” and “the”include plural references unless the context clearly dictates otherwise.For example, the term “an agent” includes a plurality of agents,including mixtures thereof.

An individual is not limited to a human being but may also be otherorganisms including, but not Limited to, mammals, plants, bacteria, orcells derived from any of the above.

Throughout this disclosure, various aspects of this invention can bepresented in a range format. It should be understood that when adescription is provided in range format, this is merely for convenienceand brevity and should not be construed as an inflexible limitation onthe scope of the invention. Accordingly, the description of a rangeshould be considered to have specifically disclosed all the possiblesub-ranges as well as individual numerical values within that range. Forexample, description of a range such as from 1 to 6 should be consideredto have specifically disclosed sub-ranges such as from 1 to 3, from 1 to4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, for example, aswell as individual numbers within that range, for example, 1, 2, 3, 4,5, and 6. This applies regardless of the breadth of the range.

The practice of the invention described herein may employ, unlessotherwise indicated, conventional techniques and descriptions of organicchemistry, polymer technology, molecular biology (including recombinanttechniques), cell biology, biochemistry, and immunology, which arewithin the skill of one of skill in the art. Such conventionaltechniques include polymer array synthesis, hybridization, ligation, anddetection of hybridization using a detectable label. Specificillustrations of suitable techniques are provided by reference to theexamples hereinbelow. However, other equivalent conventional proceduresmay also be employed. Such conventional techniques and descriptions maybe found in standard laboratory manuals, such as Genome Analysis: ALaboratory Manual Series (Vols. I-IV), Using Antibodies: A LaboratoryManual, Cells: A Laboratory Manual, PCR Primer: A Laboratory Manual, andMolecular Cloning: A Laboratory Manual (all from Cold Spring HarborLaboratory Press), Stryer, L. (1995), Biochemistry, 4th Ed., Freeman,New York, Gait, Oligonucleotide Synthesis: A Practical Approach, (1984),IRL Press, London, Nelson and Cox (2000), Lehninger, Principles ofBiochemistry, 3^(rd) Ed., W.H. Freeman Pub., New York, N.Y., and Berg etal. (2002), Biochemistry, 5^(th) Ed., W.H. Freeman Pub., New York, N.Y.,all of which are herein incorporated in their entirety by reference forall purposes.

The invention may employ solid substrates, ipcluding arrays in someembodiments. Methods and techniques applicable to polymer (includingprotein) array synthesis have been described in U.S. Ser. No. 09/536,841(abandoned), WO 00/58516, U.S. Pat. Nos. 5,143,854, 5,242,974,5,252,743, 5,324,633, 5,384,261, 5,405,783, 5,424,186, 5,451,683,5,482,867, 5,491,074, 5,527,681, 5,550,215, 5,571,639, 5,578,832,5,593,839, 5,599,695, 5,624,711, 5,631,734, 5,795,716, 5,831,070,5,837,832, 5,856,101, 5,858,659, 5,936,324, 5,968,740, 5,974,164,5,981,185, 5,981,956, 6,025,601, 6,033,860, 6,040,193, 6,090,555,6,136,269, 6,269,846 and 6,428,752, and in PCT Applications Nos.PCT/US99/00730 (International Publication No. WO 99/36760) andPCT/US01/04285 (International Publication No. WO 01/58593), which areall incorporated herein by reference in their entirety for all purposes.

Patents that describe synthesis techniques in specific embodimentsinclude U.S. Pat. Nos. 5,412,087, 6,147,205, 6,262,216, 6,310,189,5,889,165, and 5,959,098. Nucleic acid arrays are described in many ofthe above patents, but the same techniques are applied to polypeptidearrays.

Nucleic acid arrays that are useful in the described invention include,but are not limited to, those that are commercially available fromAffymetrix (Santa Clara, Calif.) under the brand name GENECHIP®. Examplearrays are shown on the Affymetrix website at the URL affymetrix.com.

Many uses for polymers attached to solid substrates have been reported.These uses include, but are not limited to, gene expression monitoring,profiling, library screening, genotyping and diagnostics. Methods ofgene expression monitoring and profiling are described in U.S. Pat. Nos.5,800,992, 6,013,449, 6,020,135, 6,033,860, 6,040,138, 6,177,248 and6,309,822. Genotyping methods, and uses thereof, are disclosed in U.S.patent application Ser. No. 10/442,021 (abandoned) and U.S. Pat. Nos.5,856,092, 6,300,063, 5,858,659, 6,284,460, 6,361,947, 6,368,799,6,333,179, and 6,872,529. Other uses are described in U.S. Pat. Nos.5,871,928, 5,902,723, 6,045,996, 5,541,061, and 6,197,506.

Also reported in the literature are various sample preparation methods.Prior to, or concurrent with, genotyping, the genomic sample may beamplified by a variety of mechanisms, some of which may employ PCR.(See, for example, PCR Technology: Principles and Applications for DNAAmplification, Ed. H. A. Erlich, Freeman Press, NY, NY, 1992; PCRProtocols: A Guide to Methods and Applications, Eds. Innis, et al.,Academic Press, San Diego, Calif., 1990; Mattila et al., Nucleic AcidsRes., 19:4967, 1991; Eckert et al., PCR Methods and Applications, 1:17,1991; PCR, Eds. McPherson et al., IRL Press, Oxford, 1991; and U.S. Pat.Nos. 4,683,202, 4,683,195, 4,800,159 4,965,188, and 5,333,675, each ofwhich is incorporated herein by reference in their entireties for allpurposes. The sample may also be amplified on the array. (See, forexample, U.S. Pat. No. 6,300,070 and U.S. patent application Ser. No.09/513,300 (abandoned), all of which are incorporated herein byreference).

Other suitable amplification methods include the ligase chain reaction(LCR) (see, for example, Wu and Wallace, Genomics, 4:560 (1989),Landegren et al., Science, 241:1077 (1988) and Barringer et al., Gene,89:117 (1990)), transcription amplification (Kwoh et al., Proc. Natl.Acad. Sci. USA, 86:1173 (1989) and WO 88/10315), self-sustained sequencereplication (Guatelli et al., Proc. Nat. Acad. Sci. USA, 87:1874 (1990)and WO 90/06995), selective amplification of target polynucleotidesequences (U.S. Pat. No. 6,410,276), consensus sequence primedpolymerase chain reaction (CP-PCR) (U.S. Pat. No. 4,437,975),arbitrarily primed polymerase chain reaction (AP-PCR) (U.S. Pat. Nos.5,413,909 and 5,861,245) and nucleic acid based sequence amplification(NABSA). (See also, U.S. Pat. Nos. 5,409,818, 5,554,517, and 6,063,603,each of which is incorporated herein by reference). Other amplificationmethods that may be used are described in, for instance, U.S. Pat. Nos.6,582,938, 5,242,794, 5,494,810, and 4,988,617, each of which isincorporated herein by reference.

Additional sample preparation methods and techniques for reducing thecomplexity of a nucleic sample are described in Dong et al., GenomeResearch, 11:1418 (2001), U.S. Pat. Nos. 6,361,947, 6,391,592,6,632,611, 6,872,529 and 6,958,225, and in U.S. patent application Ser.No. 09/916,135 (abandoned).

Methods for conducting polynucleotide hybridization assays have beenwell developed. Hybridization assay procedures and conditions will varydepending on the application and are selected in accordance with knowngeneral binding methods, including those referred to in Maniatis et al.,Molecular Cloning: A Laboratory Manual, 2^(nd) Ed., Cold Spring Harbor,N.Y, (1989); Berger and Kimmel, Methods in Enzymology, Guide toMolecular Cloning Techniques, Vol. 152, Academic Press, Inc., San Diego,Calif. (1987); Young and Davism, Proc. Nat'l. Acad. Sci., 80:1194(1983). Methods and apparatus for performing repeated and controlledhybridization reactions have been described in, for example, U.S. Pat.Nos. 5,871,928, 5,874,219, 6,045,996, 6,386,749, and 6,391,623 each ofwhich are incorporated herein by reference.

Signal detection of hybridization between ligands has been reported.(See, U.S. Pat. Nos. 5,143,854, 5,578,832, 5,631,734, 5,834,758,5,936,324, 5,981,956, 6,025,601, 6,141,096, 6,185,030, 6,201,639,6,218,803, and 6,225,625, U.S. patent application Ser. No. 10/389,194(U.S. Patent Application Publication No. 2004/0012676, allowed) and PCTApplication PCT/US99/06097 (published as WO 99/47964), each of which ishereby incorporated by reference in its entirety for all purposes).

Methods and apparatus for signal detection and processing of intensitydata are disclosed in, for example, U.S. Pat. Nos. 5,143,854, 5,547,839,5,578,832, 5,631,734, 5,800,992, 5,834,758; 5,856,092, 5,902,723,5,936,324, 5,981,956, 6,025,601, 6,090,555, 6,141,096, 6,185,030,6,201,639; 6,218,803; and 6,225,625, in U.S. Patent Pub. Nos.2004-0012676 2005-0057676 and in PCT Application PCT/US99/06097(published as WO99/47964), each of which also is hereby incorporated byreference in its entirety for all purposes.

The practice of the inventions herein may also employ conventionalbiology methods, software and systems. Computer software products of theinvention typically include, for instance, computer readable mediumhaving computer-executable instructions for performing the logic stepsof the method of the invention thereon. Suitable computer readablemedium include, but are not limited to, a floppy disk,CD-ROM/DVD/DVD-ROM, hard-disk drive, flash memory, ROM/RAM, magnetictapes, and others commonly used in the computer arts. The computerexecutable instructions may be written in a suitable computer languageor combination of several computer languages. Basic computationalbiology methods which may be employed in the invention are described in,for example, Setubal and Meidanis et al., Introduction to ComputationalBiology Methods, PWS Publishing Company, Boston, (1997); Salzberg,Searles, Kasif, (Ed.), Computational Methods in Molecular Biology,Elsevier, Amsterdam, (1998); Rashidi and Buehler, Bioinformatics Basics:Application in Biological Science and Medicine, CRC Press, London,(2000); and Ouelette and Bzevanis Bioinformatics: A Practical Guide forAnalysis of Gene and Proteins, Wiley & Sons, Inc., 2^(nd) ed., (2001).(See also, U.S. Pat. No. 6,420,108).

Various computer program products and software exist for a variety ofpurposes, such as probe design, management of data, analysis, andinstrument operation. (See, U.S. Pat. Nos. 5,593,839, 5,795,716,5,733,729, 5,974,164, 6,066,454, 6,090,555, 6,185,561, 6,188,783,6,223,127, 6,229,911 and 6,308,170). Computer methods related togenotyping using high density microarray analysis may also be used inthe present methods, see, for example, US Patent Pub. Nos. 20050250151,20050244883, 20050108197, 20050079536 and 20050042654.

Methods for analysis of genotype array data are described, for example,in Di, X., et al. (2005) Bioinformatics, 21, 1958-1963, Liu, W., et al.(2003) Bioinformatics, 19, 2397-2403 and Rabbee and Speed (2006)Bioinformatics 22:7-12. Methods for copy number analysis based onhybridization to arrays of oligonucleotides have been disclosed, forexample, in US Patent Pub. Nos. 20040157243, 20060134674, 20050130217,and 20050064476.

Additionally, the invention encompasses embodiments that may includemethods for providing genetic information over networks such as theinternet, as disclosed in, for instance, U.S. patent application Ser.No. 10/197,621 (U.S. Patent Application Publication No. 20030097222),Ser. No. 10/063,559 (U.S. Patent Application Publication No.20020183936, abandoned), Ser. No. 10/065,856 (U.S. Patent ApplicationPublication No. 20030100995, abandoned), Ser. No. 10/065,868 (U.S.Patent Application Publication No. 20030120432, abandoned), Ser. No.10/328,818 (U.S. Patent Application Publication No. 20040002818,abandoned), Ser. No. 10/328,872 (U.S. Patent Application Publication No.20040126840, abandoned), Ser. No. 10/423,403 (U.S. Patent ApplicationPublication No. 20040049354, abandoned), and 60/482,389 (expired).

II. Definition of Selected Terms

The term “may” or “microarray” as used herein refers to an intentionallycreated collection of molecules which can be prepared eithersynthetically or biosynthetically. The molecules in the array can beidentical or different from each other. The array can assume a varietyof formats, e.g., libraries of soluble molecules; libraries of compoundstethered to resin beads, silica chips, or other solid supports.Preferred arrays typically comprise a plurality of different nucleicacid probes that are coupled to a surface of a substrate in different,known locations. These arrays, also described as “microarrays” orcolloquially “chips” have been generally described in the art, forexample, U.S. Pat. Nos. 5,143,854, 5,445,934, 5,744,305, 5,677,195,5,800,992, 6,040,193, 5,424,186 and Fodor et al., Science, 251:767-777(1991). Each of which is incorporated by reference in its entirety forall purposes.

Arrays may generally be produced using a variety of techniques, such asmechanical synthesis methods or light directed synthesis methods thatincorporate a combination of photolithographic methods and solid phasesynthesis methods. Techniques for the synthesis of these arrays usingmechanical synthesis methods are described in, e.g., U.S. Pat. Nos.5,384,261, and 6,040,193, which are incorporated herein by reference intheir entirety for all purposes. Although a planar array surface ispreferred, the array may be fabricated on a surface of virtually anyshape or even a multiplicity of surfaces. Arrays may be nucleic acids onbeads, gels, polymeric surfaces, fibers such as optical fibers, glass orany other appropriate substrate. (See U.S. Pat. Nos. 5,770,358,5,789,162, 5,708,153, 6,040,193 and 5,800,992, which are herebyincorporated by reference in their entirety for all purposes.)

Many arrays are commercially available from Affymetrix under the brandname GENECHIP® and are directed to a variety of purposes, includinggenotyping and gene expression monitoring for a variety of eukaryoticand prokaryotic species. (See Affymetrix Inc., Santa Clara and theirwebsite at affymetrix.com.) Methods for preparing a genetic sample forhybridization to an array and conditions for hybridization are disclosedin the manuals provided with the arrays, for example, for expressionarrays the GENECHIP® Expression Analysis Technical Manual (PN 701021Rev. 5) provides detailed instructions for 3′ based assays and theGENECHIP® Whole Transcript (WT) Sense Target Labeling Assay Manual (PN701880 Rev. 2) provides whole transcript based assays. The GENECHIP®Mapping 100K Assay Manual (PN 701694 Rev. 3) provides detailedinstructions for sample preparation, hybridization and analysis usinggenotyping arrays. Each of these manuals is incorporated herein byreference in its entirety.

An “allele” refers to one specific form of a genetic sequence (such as agene) within a cell, an individual or within a population, the specificform differing from other forms of the same gene in the sequence of atleast one, and frequently more than one, variant sites within thesequence of the gene. The sequences at these variant sites that differbetween different alleles are termed “variances”, “polymorphisms”, or“mutations”. At each autosomal specific chromosomal location or “locus”an individual possesses two alleles, one inherited from one parent andone from the other parent, for example one from the mother and one fromthe father. An individual is “heterozygous” at a locus if it has twodifferent alleles at that locus. An individual is “homozygous” at alocus if it has two identical alleles at that locus.

The term “chromosome” refers to the heredity-bearing gene carrier of aliving cell which is derived from chromatin and which comprises DNA andprotein components (especially histones). The conventionalinternationally recognized individual human genome chromosome numberingsystem is employed herein. The size of an individual chromosome can varyfrom one type to another with a given multi-chromosomal genome and fromone genome to another. In the case of the human genome, the entire DNAmass of a given chromosome is usually greater than about 100,000,000base pairs (bp). For example, the size of the entire human genome isabout 3×10⁹ bp. The largest chromosome, chromosome no. 1, contains about2.4×10⁸ bp while the smallest chromosome, chromosome no. 22, containsabout 5.3×10⁷ bp.

The term “complementary” as used herein refers to the hybridization orbase pairing between nucleotides or nucleic acids, such as, forinstance, between the two strands of a double stranded DNA molecule orbetween an oligonucleotide primer and a primer binding site on a singlestranded nucleic acid to be sequenced or amplified. Complementarynucleotides are, generally, A and T (or A and U), or C and G. Two singlestranded RNA or DNA molecules are said to be complementary when thenucleotides of one strand, optimally aligned and compared and withappropriate nucleotide insertions or deletions, pair with at least about80% of the nucleotides of the other strand, usually at least about 90%to 95%, and more preferably from about 98 to 100%. Alternatively,complementarity exists when an RNA or DNA strand will hybridize underselective hybridization conditions to its complement. Typically,selective hybridization will occur when there is at least about 65%complementary over a stretch of at least 14 to 25 nucleotides,preferably at least about 75%, more preferably at least about 90%complementary. (See, M. Kanehisa Nucleic Acids Res. 12:203 (1984),incorporated herein by reference).

The term “complex population or mixed population” as used herein refersto any sample containing both desired and undesired nucleic acids. As anon-limiting example, a complex population of nucleic acids may be totalgenomic DNA, total genomic RNA or a combination thereof. Moreover, acomplex population of nucleic acids may have been enriched for a givenpopulation but include other undesirable populations. For example, acomplex population of nucleic acids may be a sample which has beenenriched for desired messenger RNA (mRNA) sequences but still includessome undesired ribosomal RNA sequences (rRNA).

The term “effective amount” as used herein refers to an amountsufficient to induce a desired result.

The term “genome” as used herein is all the genetic material in thechromosomes of an organism. DNA derived from the genetic material in thechromosomes of a particular organism is genomic DNA. A genomic libraryis a collection of clones made from a set of randomly generatedoverlapping DNA fragments representing the entire genome of an organism.

The term “genotyping” refers to the determination of the geneticinformation an individual carries at one or more positions in thegenome. For example, genotyping may comprise the determination of whichallele or alleles an individual carries for a single SNP or thedetermination of which allele or alleles an individual carries for aplurality of SNPs. For example, a particular nucleotide in a genome maybe an A in some individuals and a C in other individuals. Thoseindividuals who have an A at the position have the A allele and thosewho have a C have the C allele. In a diploid organism the individualwill have two copies of the sequence containing the polymorphic positionso the individual may have an A allele and a C allele or alternativelytwo copies of the A allele or two copies of the C allele. Thoseindividuals who have two copies of the C allele are homozygous for the Callele, those individuals who have two copies of the A allele arehomozygous for the C allele, and those individuals who have one copy ofeach allele are heterozygous. The array may be designed to distinguishbetween each of these three possible outcomes. A polymorphic locationmay have two or more possible alleles and the array may be designed todistinguish between all possible combinations.

The term “hybridization conditions” as used herein will typicallyinclude salt concentrations of less than about 1 M, more usually lessthan about 500 mM and preferably less than about 200 mM. Hybridizationtemperatures can be as low as 5° C., but are typically greater than 22°C., more typically greater than about 30° C., and preferably in excessof about 37° C. Longer fragments may require higher hybridizationtemperatures for specific hybridization. As other factors may affect thestringency of hybridization, including base composition and length ofthe complementary strands, presence of organic solvents and extent ofbase mismatching, the combination of parameters is more important thanthe absolute measure of any one alone.

The term “hybridization” as used herein refers to the process in whichtwo single-stranded polynucleotides bind non-covalently to form a stabledouble-stranded polynucleotide; triple-stranded hybridization is alsotheoretically possible. The resulting (usually) double-strandedpolynucleotide is a “hybrid.” The proportion of the population ofpolynucleotides that forms stable hybrids is referred to herein as the“degree of hybridization.” Hybridizations are usually performed understringent conditions, for example, at a salt concentration of no morethan 1 M and a temperature of at least 25° C. For example, conditions of5×SSPE (750 mM NaCl, 50 mM NaPhosphate, 5 mM EDTA, pH 7.4) and atemperature of 25-30° C. are suitable for allele-specific probehybridizations. For stringent conditions, see, for example, Sambrook,Fritsache and Maniatis, “Molecular Cloning A laboratory Manual” 2^(nd)Ed. Cold Spring Harbor Press (1989), which is hereby incorporated byreference in its entirety for all purposes above. Hybridizations, e.g.,allele-specific probe hybridizations, are generally performed understringent conditions. For example, conditions where the saltconcentration is no more than about 1 Molar (M) and a temperature of atleast 25° C., e.g., 750 mM NaCl, 50 mM Sodium Phosphate, 5 mM EDTA, pH7.4 (5×SSPE) and a temperature of from about 25 to about 30° C.

The term “hybridization probes” as used herein are oligonucleotidescapable of binding in a base-specific manner to a complementary strandof nucleic acid. Such probes include peptide nucleic acids, as describedin Nielsen et al., Science, 254, 1497-1500 (1991), and other nucleicacid analogs and nucleic acid mimetics.

The term “hybridizing specifically to” as used herein refers to thebinding, duplexing, or hybridizing of a molecule only to a particularnucleotide sequence or sequences under stringent conditions when thatsequence is present in a complex mixture (e.g., total cellular) DNA orRNA.

The term “isolated nucleic acid” as used herein mean an object speciesinvention that is the predominant species present (e.g., on a molarbasis it is more abundant than any other individual species in thecomposition). Preferably, an isolated nucleic acid comprises at leastabout 50, 80 or 90% (on a molar basis) of all macromolecular speciespresent. Most preferably, the object species is purified to essentialhomogeneity (contaminant species cannot be detected in the compositionby conventional detection methods).

The term “ligand” as used herein refers to a molecule that is recognizedby a particular receptor. The agent bound by or reacting with a receptoris called a “ligand,” a term which is definitionally meaningful only interms of its counterpart receptor. The term “ligand” does not imply anyparticular molecular size or other structural or compositional featureother than that the substance in question is capable of binding orotherwise interacting with the receptor. Also, a ligand may serve eitheras the natural ligand to which the receptor binds, or as a functionalanalogue that may act as an agonist or antagonist. Examples of ligandsthat can be investigated by this invention include, but are notrestricted to, agonists and antagonists for cell membrane receptors,toxins and venoms, viral epitopes, hormones (e.g., opiates, steroids,and other similar compounds), hormone receptors, peptides, enzymes,enzyme substrates, substrate analogs, transition state analogs,cofactors, drugs, proteins, and antibodies.

The term “linkage disequilibrium” or “allelic association” as usedherein refers to the preferential association of a particular allele orgenetic marker with a specific allele, or genetic marker at a nearbychromosomal location more frequently than expected by chance for anyparticular allele frequency in the population. For example, if locus Xhas alleles a and b, which occur equally frequently, and linked locus Yhas alleles c and d, which occur equally frequently, one would expectthe combination ac to occur with a frequency of 0.25. If ac occurs morefrequently, then alleles a and c are in linkage disequilibrium. Linkagedisequilibrium may result from natural selection of certain combinationof alleles or because an allele has been introduced into a populationtoo recently to have reached equilibrium with linked alleles.

The term “mixed population” as used herein refers to a complexpopulation.

The term “monomer” as used herein refers to any member of the set ofmolecules that can be joined together to form an oligomer or polymer.The set of monomers useful in this invention includes, but is notrestricted to, for the example of (poly)peptide synthesis, the set ofL-amino acids, D-amino acids, or synthetic amino acids. As used herein,“monomer” refers to any member of a basis set for synthesis of anoligomer. For example, dimers of L-amino acids form a basis set of 400“monomers” for synthesis of polypeptides. Different basis sets ofmonomers may be used at successive steps in the synthesis of a polymer.The term “monomer” also refers to a chemical subunit that can becombined with a different chemical subunit to form a compound largerthan either subunit alone.

The term “b” or “mRA transcript” as used herein, include, but notlimited to pre-message RNA transcript(s), transcript processingintermediates, mature mRNA(s) ready for translation and transcripts ofthe gene or genes, or nucleic acids derived from the mRNA transcript(s).Transcript processing may include splicing, editing and degradation. Asused herein, a nucleic acid derived from an mRNA transcript refers to anucleic acid for whose synthesis the mRNA transcript or a subsequencethereof has ultimately served as a template. Thus, a cDNA reversetranscribed from an mRNA, an RNA transcribed from that cDNA, a DNAamplified from the cDNA, an RNA transcribed from the amplified DNA, forexample, are all derived from the mRNA transcript and detection of suchderived products is indicative of the presence and/or abundance of theoriginal transcript in a sample. Thus, mRNA derived samples include, butare not limited to, mRNA transcripts of the gene or genes, cDNA reversetranscribed from the mRNA, cRNA transcribed from the cDNA, DNA amplifiedfrom the genes, RNA transcribed from amplified DNA, and the like.

The term “nucleic acid library or army” as used herein refers to anintentionally created collection of nucleic acids which can be preparedeither synthetically or biosynthetically and screened for biologicalactivity in a variety of different formats (e.g., libraries of solublemolecules; and libraries of oligos tethered to resin beads, silicachips, or other solid supports). Additionally, the term “array” is meantto include those libraries of nucleic acids which can be prepared byspotting nucleic acids of essentially any length (e.g., from 1 to about1000 nucleotide monomers in length) onto a substrate. The term “nucleicacid” as used herein refers to a polymeric form of nucleotides of anylength, either ribonucleotides, deoxyribonucleotides, locked nucleicacids (LNAs) or peptide nucleic acids (PNAs), that comprise purine andpyrimidine bases, or other natural, chemically or biochemicallymodified, non-natural, or derivatized nucleotide bases. The backbone ofthe polynucleotide can comprise sugars and phosphate groups, as maytypically be found in RNA or DNA, or modified or substituted sugar orphosphate groups. A polynucleotide may comprise modified nucleotides,such as methylated nucleotides and nucleotide analogs. The sequence ofnucleotides may be interrupted by non-nucleotide components. Thus theterms nucleoside, nucleotide, deoxynucleoside and deoxynucleotidegenerally include analogs such as those described herein. These analogsare those molecules having some structural features in common with anaturally occurring nucleoside or nucleotide such that when incorporatedinto a nucleic acid or oligonucleoside sequence, they allowhybridization with a naturally occurring nucleic acid sequence insolution. Typically, these analogs are derived from naturally occurringnucleosides and nucleotides by replacing and/or modifying the base, theribose or the phosphodiester moiety. The changes can be tailor made tostabilize or destabilize hybrid formation or enhance the specificity ofhybridization with a complementary nucleic acid sequence as desired.

The term “nucleic acids” as used herein may include any polymer oroligomer of pyrimidine and purine bases, preferably cytosine, thymine,and uracil, and adenine and guanine, respectively. (See, Albert L.Lehninger, “Principles of Biochemistry,” at 793-800, Worth Pub. 1982).Indeed, the invention contemplates any deoxyribonucleotide,ribonucleotide or peptide nucleic acid component, and any chemicalvariants thereof, such as methylated, hydroxymethylated or glucosylatedforms of these bases, and the like. The polymers or oligomers may beheterogeneous or homogeneous in composition, and may be isolated fromnaturally-occurring sources or may be artificially or syntheticallyproduced. In addition, the nucleic acids may be DNA or RNA, or a mixturethereof, and may exist permanently or transitionally in single-strandedor double-stranded form, including homoduplex, heteroduplex, and hybridstates.

The term “oligonucleotide” or “polynucleotide” as used interchangeablyherein refers to a nucleic acid ranging from at least 2, preferable atleast 8, and more preferably at least 20 nucleotides in length or acompound that specifically hybridizes to a polynucleotide.Polynucleotides of the invention include sequences of deoxyribonucleicacid (DNA) or ribonucleic acid (RNA) which may be isolated from naturalsources, recombinantly produced or artificially synthesized and mimeticsthereof. A further example of a polynucleotide of the invention may belocked nucleic acids (LNAs) or peptide nucleic acid (PNA). The inventionalso encompasses situations in which there is a nontraditional basepairing such as Hoogsteen base pairing which has been identified incertain tRNA molecules and postulated to exist in a triple helix.“Polynucleotide” and “oligonucleotide” are used interchangeably in thisapplication.

The term “robe” as used herein refers to a surface-immobilized moleculethat can be recognized by a particular target. See U.S. Pat. No.6,582,908 for an example of arrays having all possible combinations ofprobes with 10, 12, and more bases. Examples of probes that can beinvestigated by this invention include, but are not restricted to,agonists and antagonists for cell membrane receptors, toxins and venoms,viral epitopes, hormones (e.g., opioid peptides, steroids), hormonereceptors, peptides, enzymes, enzyme substrates, cofactors, drugs,lectins, sugars, oligonucleotides, nucleic acids, oligosaccharides,proteins, and monoclonal antibodies.

The term “primer” as used herein refers to a single-strandedoligonucleotide capable of acting as a point of initiation fortemplate-directed DNA synthesis under suitable conditions e.g., bufferand temperature, in the presence of four different nucleosidetriphosphates and an agent for polymerization, such as, for example, DNAor RNA polymerase or reverse transcriptase. The length of the primer, inany given case, depends on, for example, the intended use of the primer,and generally ranges from 15 to 30 nucleotides. Short primer moleculesgenerally require cooler temperatures to form sufficiently stable hybridcomplexes with the template. A primer need not reflect the exactsequence of the template but must be sufficiently complementary tohybridize with such template. The primer site is the area of thetemplate to which a primer hybridizes. The primer pair is a set ofprimers including a 5′ upstream primer that hybridizes with the 5′ endof the sequence to be amplified and a 3′ downstream primer thathybridizes with the complement of the 3′ end of the sequence to beamplified.

The term “polymorphism” as used herein refers to the occurrence of twoor more genetically determined alternative sequences or alleles in apopulation. A polymorphic marker or site is the locus at whichdivergence occurs. Preferred markers have at least two alleles, eachoccurring at frequency of greater than 1%, and more preferably greaterthan 10% or 20% of a selected population. A polymorphism may compriseone or more base changes, an insertion, a repeat, or a deletion. Apolymorphic locus may be as small as one base pair. Polymorphic markersinclude restriction fragment length polymorphisms, variable number oftandem repeats (VNTR's), hypervariable regions, minisatellites,dinucleotide repeats, trinucleotide repeats, tetranucleotide repeats,simple sequence repeats, and insertion elements such as Alu. The firstidentified allelic form is arbitrarily designated as the reference formand other allelic forms are designated as alternative or variantalleles. The allelic form occurring most frequently in a selectedpopulation is sometimes referred to as the wildtype form. Diploidorganisms may be homozygous or heterozygous for allelic forms. Adiallelic polymorphism has two forms. A triallelic polymorphism hasthree forms. Single nucleotide polymorphisms (SNPs) are included inpolymorphisms.

The term “solid support”, “support”, and “substrate” as used herein areused interchangeably and refer to a material or group of materialshaving a rigid or semi-rigid surface or surfaces. In many embodiments,at least one surface of the solid support will be substantially flat,although in some embodiments it may be desirable to physically separatesynthesis regions for different compounds with, for example, wells,raised regions, pins, etched trenches, or the like. According to otherembodiments, the solid support(s) will take the form of beads, resins,gels, microspheres, or other geometric configurations. (See, U.S. Pat.No. 5,744,305 for exemplary substrates).

The term “target” as used herein refers to a molecule that has anaffinity for a given probe. Targets may be naturally-occurring orman-made molecules. Also, they can be employed in their unaltered stateor as aggregates with other species. Targets may be attached, covalentlyor noncovalently, to a binding member, either directly or via a specificbinding substance. Examples of targets which can be employed by thisinvention include, but are not restricted to, antibodies, cell membranereceptors, monoclonal antibodies and antisera reactive with specificantigenic determinants (such as on viruses, cells or other materials),drugs, oligonucleotides, nucleic acids, peptides, cofactors, lectins,sugars, polysaccharides, cells, cellular membranes, and organelles.Targets are sometimes referred to in the art as anti-probes. As the termtargets is used herein, no difference in meaning is intended. A “ProbeTarget Pair” is formed when two macromolecules have combined throughmolecular recognition to form a complex.

A “translocation” or “chromosomal translocation” is a chromosomeabnormality caused by rearrangement of parts between nonhomologouschromosomes. It is detected on cytogenetics or a karyotype of affectedcells. There are two main types, reciprocal (also known asnon-Robertsonian) and Robertsonian. Also, translocations can be balanced(in an even exchange of material with no genetic information extra ormissing, and ideally full functionality) or unbalanced (where theexchange of chromosome material is unequal resulting in extra or missinggenes).

A karyotype is the observed characteristics (number, type, shape etc) ofthe chromosomes of an individual or species.

In normal diploid organisms, autosomal chromosomes are present in twoidentical copies, although polyploid cells have multiple copies ofchromosomes and haploid cells have single copies. The chromosomes arearranged and displayed (often on a photo) in a standard format known asan idiogram: in pairs, ordered by size and position of centromere forchromosomes of the same size. Karyotypes are used to study chromosomalaberrations, and may be used to determine other macroscopically visibleaspects of an individual's genotype, such as sex. In order to be able tosee the chromosomes and determine their size and internal pattern, theyare chemically labeled with a dye (“stained”). The pattern of individualchromosomes is called chromosome banding.

Normal human karyotypes contain 22 pairs of autosomal chromosomes andone pair of sex chromosomes. Normal karyotypes for women contain two Xchromosomes and are denoted 46,XX; men have both an X and a Y chromosomedenoted 46,XY.

A “single-nucleotide polymorphism” (SNP) is a DNA sequence variationoccurring when a single nucleotide—A, T, C, or G—in the genome (or othershared sequence) differs between members of a species (or between pairedchromosomes in an individual). For example, two sequenced DNA fragmentsfrom different individuals, AAGCCTA to AAGCTTA, contain a difference ina single nucleotide. In this case we say that there are two alleles: Cand T. Almost all common SNPs have only two alleles.

Loss of Heterozygosity (LOH) represents the loss of normal function ofone allele of a gene in which the other allele was already inactivated.In oncology, loss of heterozygosity occurs when the remaining functionalallele in a somatic cell of the offspring becomes inactivated bymutation. This results in no normal tumor suppressor being produced andthis could result in tumorigenesis. Zygosity is the similarity of genesfor a trait (inherited characteristic) in an organism. If both genes arethe same, the organism is homozygous for the trait. If both genes aredifferent, the organism is hctcrozygous for that trait. If one gene ismissing, it is hemizygous, and if both genes are missing, it isnullizygous. The DNA sequence of any gene can vary among individuals inthe population. The various forms of a gene are called alleles, anddiploid organisms generally have two alleles for each gene, one on eachof the two homologous chromosomes on which the gene is present. Indiploid organisms, the alleles are inherited from the individual'sparents, one from the male parent and one from the female. Zygosity ingeneral is a description of whether those two alleles have identical ordifferent DNA sequences. For cytogenetists, detection of LOH isimportant because such genetic abnormalities may be associated withgenetic disorders.

In particular, a major focus in cytogenetics research is on UniparentalDisomy (UPD) events where a child inherits two copies of chromosomalmaterial from one parent and nothing from the other. These UPD eventsare known to be linked with recessive disorders and also causedevelopmental disorders due to gene imprinting. These events occurwithout associated copy number changes. For instance approximately 30%of Prader-Willi cases are associated with paternal UPD of chromosome15q, 2-3% of Angelman Syndrome are associated with maternal UPD of 15q,10-30% of Beckwith-Wiedemann Syndrome are associated with maternal UPDof 11p15, and 5% of Silver-Russell Syndrome are associated with maternalUPD of chromosome 7.

LOH is also known to be associated with consanguinity and inbreeding.The boundary between the two of these genetic events is not alwaysclear. Generally, consanguinity refers to close relation matingsproducing off-spring, e.g. first cousin pairings. This will tend toresult in large blocks of LOH, perhaps on only a few chromosomes.Inbreeding refers to small isolated (perhaps physically or culturally)populations where the degree of genetic variability is low within thepopulation. This may lead to many small blocks of LOH across manychromosomes.

Lone Contiguous Stretches of Homozygosity (LCSH) in a genomic region(stretch) indicates a region in which the Copy Number is neutral (twocopies) but which displays a Loss of normal heterozygosity, and thus ishomozygous for the measured SNP allele information.

The term “copy number” (CN) refers to the number of copies of theparticular gene or fragment of a gene being measured or detected in agenetic sample. Normal copy number for organisms is two, i.e. there aretwo copies of each gene present in every cell. However, most organismspossess a variety of numbers of copies of genes in their cells at anygiven time. Copy number may be increased in some cancers or diseases, ordecreased.

A copy number variation (CNV) is a segment of DNA in which copy numberdifferences have been found by comparison of two or more genomes. Thesegment may range from one kilobase to several megabases in size. Humans(being normally diploid) ordinarily have two copies of each autosomalregion of genetic material, one per chromosome. This may vary forparticular genetic regions due to deletion or duplication events. CNVsmay either be inherited or caused by de novo mutation. CNVs can becaused by genomic rearrangements such as deletions, duplications,inversions, and translocations.

Low copy repeats (LCRs), which are region-specific repeat sequences, aresusceptible to such genomic rearrangements resulting in CNVs. Factorssuch as size, orientation, percentage similarity and the distancebetween the gene copies renders them susceptible.

Copy Number Polymorphism (CNP) analysis is a specialized method fordetermining the copy number state in specific genomic CNP regions. CNPregions are observed to be more variable in copy number state than thegenome as a whole. Because the copy number state in CNP regions is morelikely to deviate from the normal copy number state of two, previouscopy number analysis methods were less accurate when applied to CNPregions. Therefore, a new analysis algorithm called Canary was developedby researchers at the Broad Institute for copy number analysis withinCNP regions.

An Annotation track provides information about the genetic code to whichit is attached. For instance, an annotation track may provide the userwith visual information indicating whether a selected segment of thegenome displays LOH, LCSH or any other such genetic characteristic orabnormality.

A Hidden Markov Model (HMM) is a statistical model where the systembeing modeled is assumed to be a Markov process with unknown parameters,and the challenge is to determine the hidden parameters from theobservable parameters. HMM statistical models are used by the disclosedinvention software application to determine whether, for instance, thereis a change in Copy Number State. The extracted model parameters canthen be used to perform further analysis, for example, for patternrecognition applications. A HMM can be considered as the simplestdynamic Bayesian network. In a regular Markov model, the state isdirectly visible to the observer, and therefore the state transitionprobabilities are the only parameters. In a hidden Markov model, thestate is not directly visible, but variables influenced by the state arevisible. Each state has a probability distribution over the possibleoutput values. Therefore the sequence of values generated by an HMMgives some information about the sequence of states, e.g. Copy NumberStates. Hidden Markov models are especially known for their applicationin temporal pattern recognition such as speech, handwriting, gesturerecognition and bioinformatics. (See, for instance, Lior Pachter andBernd Sturmfels, “Algebraic Statistics for Computational Biology,”Cambridge University Press, 2005, ISBN 0-521-85700-7; Eddy, NatureBiotechnology, 22:1315-1316 (2005) and Pavel Pevzncr, “ComputationalMolecular Biology: An Algorithmic Approach,” MIT Press, 2000, especiallypp. 145-149; see also, Rabiner, L., “A Tutorial on Hidden Markov Modelsand Selected Applications in Speech Recognition,” Proceedings of theIEEE, Volume 77, pp 257-86, 1989). A HMM is typically defined by a setof hidden states, a matrix of state transition probabilities and amatrix of emission probabilities. Each hidden state has differentstatistical properties. (See, U.S. patent application Ser. No.12/143,754, corresponding to U.S. Patent Application Publication No.2009/0098547, incorporated herein by reference for purposes).Application of the HMM model to genetic data may yield, for instance, aset of reported probabilities at each defined genetic marker of whetherthe marker is either normal or abnormal in copy number. Suchprobabilities may be summarized by the disclosed software applicationfor entire segments.

Basically, the HMM processes data one chromosome at a time; a singlechromosome is input, a copy number is assigned to each marker, then themarkers are partitioned into contiguous segments. A brief summary of theprocess may be represented as follows. First, summaries are computed ateach pre-defined genetic marker based on the likelihood function of themodel. The likelihood-based summary provides the probability of themarker belonging to each of the copy number categories, but ignores thesame probabilities at neighboring markers. Second, the HMM combinesprobabilities of markers by taking into account probabilities ofneighboring markers as well as probabilities of copy number statetransitions from one marker to the next. The result is a chain of copynumber calls, one call at each marker, extending the length of thechromosome. The transition probabilities may be specified ahead of timeand act independent of the data. Third, of the many possible chains ofcopy number calls that could be simultaneously assigned to an entirechromosome of markers, the chain with the highest probability isreported to the user, and later displayed to the user by the softwareapplication. Finally, the chain of copy number calls is partitioned intosegments such that each copy number segment has an identical copy numbercall at each marker. End points of segments therefore indicate a changein copy number call.

An Expressed Sequence Tag (ST) is a short sub-sequence of a transcribedcDNA sequence. They may be used to identify gene transcripts, and areinstrumental in gene discovery and gene sequence determination. Theidentification of ESTs has proceeded rapidly, with approximately 52million ESTs available in public databases as of 2008 (e.g. GenBank, andothers).

Mosaicism, is the presence of differing genetic sequences or compositionwithin a specified region of the genome. Copy Number Mosaicism indicatesthe experimental sample genetic marker in question has a copy numberwhich is not a whole integer, but rather is a fractional number, e.g.the copy number is determined by the software and analysis to not be oneor two, but is instead determined empirically from the experimentalsample to be, for example, a value of 1.6.

MAPD is the Median Absolute Pairwise Difference statistic. MAPD isdefined as the Median of the Absolute values of all Pairwise Differencesbetween log₂ ratios for a given probe array. Each pair is defined asadjacent in terms of genomic distance, with SNP markers and CN markersbeing treated equally. Hence, any two markers that are adjacent in thegenomic coordinates are a pair. Except at the beginning and the end of achromosome, every marker belongs to two pairs as it is adjacent to amarker preceding it and a marker following it on the genome. Formally,if xi: is the log 2 ratio for marker i:

MAPD=median(|x _(i+1) −x _(i) |, i ordered by genomic position)

MAPD is a per-chip estimate of variability, like standard deviation (SD)or interquartile range (IQR). If the log 2 ratios are distributednormally with a constant SD, then MAPD/0.96 is equal to SD and MAPD*1.41is equal to IQR. However, unlike SD or IQR, using MAPD is robust againsthigh biological variability in log 2 ratios induced by conditions suchas cancer.

MAPD Weight is also a copy number parameter, which is used to add theMedian Absolute Pairwise Difference statistic to the dispersionparameter Standard Deviation found in Copy Number Parameters:HMMParameters:Priors. If the MAPD weight is increased from the default, itmakes sense to decrease the Standard Deviation.

Mean is a copy number parameter which lists the expected values of thelog base 2 ratios with respect to the reference sample corresponding toeach copy number state. It is best to have the means as accurate aspossible, however, it is difficult to estimate the copy number means ofany sample from within the sample.

Standard Deviation is a copy number parameter that lists thecorresponding expected standard deviations in the log 2 ratio datacorresponding to each copy number state. Note that MAPD is computed foreach sample and added to these standard deviations after MAPD ismultiplied by the MAPD weight.

Smoothing as used herein refers to a process, performed by a computersoftware program, of manipulating the data found in user-definedSegments (defined below). For instance, smoothing of Copy Numbersegments can take place when more than one adjacent segment has anaberrant copy number call. For example, if there is a stretch of twentymarkers with copy number three, followed by a stretch of five markerswith copy number four, followed by a stretch often markers with copynumber three. These three segments that are together are anuninterrupted copy number gain. The three segments can be smoothed intoone, in which case. Smoothing only takes place over stretches that areentirely a gain or entirely a loss. As another example, consider a setof data having a contiguous set of segments with gain values (forinstance, of CNState values of three and four), with no markers of copynumber two or lower. Without smoothing, these segments will be treatedand represented by the software application as a series of individualgain segments. The same rules apply to a set of segments with lossvalues of 0 or 1. If within the data there is present a contiguous setof markers with gain values of three and four, with no interveningmarkers of copy number two or lower, then these data will beconsolidated into a single gain segment with smoothing applied. It isimportant to note that smoothing is only a visual aid to the user anddoes not affect the actual values (data file, like a .chp file) used asthe input in the HMM process. The actual values remain unchanged in thedata file for later use.

Joining is also an optional manipulation of genetic data that can beperformed on the Copy Number Segment data. Joining can occur when two ormore otherwise contiguous aberrant copy number segments of the same copynumber are interrupted by a normal copy number segment. For example, ifthere is a stretch of 15 markers with copy number 3 is interrupted by 5markers called as normal, followed by 25 markers with copy number 3. The3 segments can be joined into one copy number gain by ignoring the shortnormal stretch. The short ignored stretch is treated as missing. Joiningoptions within the software application allow the user to join segmentswith the same aberrant CNState that are separated by no more than auser-specified number of normal-state markers, or by no more than auser-specified distance of normal-state data.

Smoothing and Joining are non-destructive mathematical processes thataffect the display of Copy Number segments in software programs.Smoothing and Joining are performed on the CNState data as it is loadedinto the software application, based on settings that the user setsbefore loading. These processes do not affect the marker data in the.cnchp/.cychp files (explained in further detail below).

Generally, confidence parameter values are generated during scanning ofthe probe array and included in the .cychp data file. This parameterindicates the length of the segment and the number of markers per unitlength. The confidence parameter therefore is a measure of thelikelihood that the segment represents a real change in the sequence ofthe genome as compared with a standard or normal or control sample. Thisconfidence score may need to be recalculated during segment detectionbased on various mathematical algorithm applications, such as smoothingor joining.

The methods may be combined with other methods of genome analysis andcomplexity reduction. Other methods of complexity reduction include, forexample, AFLP, see U.S. Pat. No. 6,045,994, which is incorporated hereinby reference, and arbitrarily primed-PCR (AP-PCR) see McClelland andWelsh, in PCR Primer: A laboratory Manual, (1995) eds. C. Dieffenbachand G. Dveksler, Cold Spring Harbor Lab Press, for example, at p 203,which is incorporated herein by reference in its entirety. Additionalmethods of sample preparation and techniques for reducing the complexityof a nucleic sample are described in Dong et al., Genome Research 11,1418 (2001), in U.S. Pat. Nos. 6,361,947, 6,391,592, 6,458,5306,872,529, 6,958,225, and 6,632,611 and U.S. Patent Pub. Nos.20030039069, 2004-0067493 and 2004-0067493, which are incorporatedherein by reference in their entireties.

The design and use of allele-specific probes for analyzing polymorphismsis described by e.g., Saiki et al., Nature 324, 163-166 (1986);Dattagupta, EP 235,726, Saiki, and WO 89/11548. Allele-specific probescan be designed that hybridize to a segment of target DNA from oneindividual but do not hybridize to the corresponding segment fromanother individual due to the presence of different polymorphic forms inthe respective segments from the two individuals. Hybridizationconditions should be sufficiently stringent that there is a significantdifference in hybridization intensity between alleles, and preferably anessentially binary response, whereby a probe hybridizes to only one ofthe alleles.

The term “covariate” is generally a statistical or mathematical termmeaning a variable which is possibly predictive of the outcome understudy. A covariate variable may be thought of as an independentvariable, as used in regression analysis. Covariate values may impactthe outcome or interpretation of data.

The term “covariate adjustor” means a computer software program which iscapable of manipulating a set of data so that a covariate value eitherhas little or no impact on the final adjusted data set or has more of animpact to better reflect reality or what is actually being detected. Forinstance, in hybridizing a labeled genetic sample to a DNA microarray,many variables exist which interfere with interpretation of the dataderived from scanning the array and reading the fluorescent intensitysignals. The manner in which the genetic sample was prepared, using PCRor other amplification processes, may favor amplification of aparticular kind of gene fragment over others, resulting in an increasedintensity of that fragment even though in the cells in which the geneticsample originated, i.e. in reality, the gene on that fragment is in factnot present in higher proportions than normal. This is a non-limitingand simple example of a covariate value which would need adjustment inthe final analysis and interpretation of the data. Covariate adjustersmay also be thought of as removing “noise” or interference fromnon-experiment-related factors, which may normally be removed in anexperimental model by various well-designed controls. Covariateadjustors mathematically remove these variables in a controlled mannersuch that the data ultimately reflect reality, i.e. what is actuallypresent inside the genetic sample. Covariate adjusters can removespurious artifacts unrelated to the genes being detected in the geneticsample.

Below are provided various embodiments explaining generally how theinvention may work and may be implemented or applied to the study ofgenetic information. Though these embodiments may be very specific, itis understood by one of skill in the art that many modifications may bemade of these specifications to achieve the same general outcome. All ofthese generally known and acknowledged alternative embodiments areincorporated herein within the scope of this disclosed invention.

III. Systems Useful in Analysis of Microarrays

This invention relates to software that accepts, analyzes and visuallypresents data obtained from nucleic acid probe microarrays, such asAFFYMETRIX® GENECHIP® probe arrays, and spotted probe arrays, asdescribed above. The data obtained from such microarray experiments istypically a number of signal intensity values obtained from scans of themicroarrays hybridized with labeled genetic test samples. Thesemicroarrays have been used to generate unprecedented amounts ofinformation about biological systems and diseases. For example, theAffymetrix Genome-Wide Human SNP Array 6.0 and CytoScan HD Array,available from Affymetrix, Inc. of Santa Clara, Calif., containsmillions of oligonucleotides probes on a single microarray andrepresents the most advanced microarray of its kind in the market.Analysis of expression and genotype data from such microarrays may leadto the development of new drugs and new diagnostic tools.

Various array configurations and machines are available which enableinterrogation and analysis of labeled genomic samples hybridized tomicroarrays. (See, for instance, US. Pat. Nos. 5,445,934; 5,744,305;5,945,334; 6,140,044; 6,261,776; 6,291,183; 6,346,413; 6,399,365;6,420,169; 6,551,817; 6,610,482; 6,733,977; 6,955,915; D430,024;5,445,934; 5,744,305; 6,261,776; 6,291,183; 6,346,413; 6,399,365;6,610,482; 6,733,977 concerning various arrays; U.S. Pat. Nos.6,114,122; 6,287,850; 6,391,623; and 6,422,249 concerning variousfluidics stations; U.S. Pat. Nos. 5,578,832; 5,631,734; 5,834,758;5,936,324; 5,981,956; 6,025,601; 6,141,096; 6,171,793; 6,185,030;6,201,639; 6,207,960; 6,218,803; 6,225,625; 6,252,236; 6,335,824;6,403,320; 6,407,858; 6,472,671; 6,490,533; 6,650,411; 6,643,015;6,813,567; 6,141,096, 6,262,838; 6,294,327; 6,403,320; 6,407,858;6,597,000; and 7,406,391 concerning various scanners; U.S. Pat. Nos.6,511,277; 6,604,902; 6,705,754; and 7,108,472 concerning variousauto-loading devices useful with the instrumentation and software of thepresent invention; all incorporated herein by reference for all purposesin their entirety).

Various techniques and technologies may be used for synthesizing densearrays of biological materials on or in a substrate or support. Forexample, the Affymetrix GENECHIP™ arrays are synthesized in accordancewith techniques sometimes referred to as VLSPS™ (Very Large ScaleImmobilized Polymer Synthesis) technologies. Some aspects of VLSPS™ andother microarray manufacturing technologies are described in U.S. Pat.Nos. 5,424,186; 5,143,854; 5,445,934; 5,744,305; 5,831,070; 5,837,832;6,022,963; 6,083,697; 6,291,183; 6,309,831; and 6,310,189, all of whichare hereby incorporated by reference in their entireties for allpurposes. The probes of these arrays in some implementations consist ofnucleic acids that are synthesized by methods including the steps ofactivating regions of a substrate and then contacting the substrate witha selected monomer solution. As used herein, nucleic acids may includeany polymer or oligomer of nucleosides or nucleotides (polynucleotidesor oligonucleotides) that include pyrimidine and/or purine bases,preferably cytosine, thymine, and uracil, and adenine and guanine,respectively. Nucleic acids may include any deoxyribonucleotide,ribonucleotide, and/or peptide nucleic acid component, and/or anychemical variants thereof such as LNAs, methylated, hydroxymethylated orglucosylated forms of these bases, and the like. The polymers oroligomers may be heterogeneous or homogeneous in composition, and may beisolated from naturally-occurring sources or may be artificially orsynthetically produced. In addition, the nucleic acids may be DNA orRNA, or a mixture thereof, and may exist permanently or transitionallyin single-stranded or double-stranded form, including homoduplex,heteroduplex, and hybrid states. Probes of other biological materials,such as peptides or polysaccharides as non-limiting examples, may alsobe formed. For more details regarding possible implementations, see U.S.Pat. No. 6,156,501, which is hereby incorporated by reference herein inits entirety for all purposes.

A system and method for efficiently synthesizing probe arrays usingmasks is described in U.S. Pat. No. 6,949,638, which is herebyincorporated by reference herein in its entirety for all purposes. Asystem and method for a rapid and flexible microarray manufacturing andonline ordering system is described in U.S. Provisional PatentApplication Ser. No. 60/265,103 (now expired), filed Jan. 29, 2001,which also is hereby incorporated herein by reference in its entiretyfor all purposes. Systems and methods for optical photolithographywithout masks are described in U.S. Pat. No. 6,271,957 and in U.S.patent application Ser. No. 09/683,374 filed Dec. 19, 2001 (nowabandoned), both of which are hereby incorporated by reference herein intheir entireties for all purposes.

Other techniques exist for depositing probes on a substrate or support.For example, “spotted arrays” are commercially fabricated, typically onmicroscope slides. Aspects of these and other spot arrayers aredescribed in U.S. Pat. Nos. 6,040,193 and 6,136,269, in U.S. Pat. No.6,955,788, and in International Patent Application No. PCT/US99/00730(International Publication Number WO 99/36760), all of which are herebyincorporated by reference in their entireties for all purposes. Othertechniques for generating spotted arrays also exist. For example, U.S.Pat. No. 6,040,193 to Winkler, et al., is directed to processes fordispensing drops to generate spotted arrays.

Labeled targets in hybridized probe arrays may be detected using variouscommercial devices, sometimes referred to as scanners. For example, ascanning system for use with a fluorescent label is described in U.S.Pat. No. 5,143,854, incorporated by reference above. Other scanners orscanning systems are described in U.S. Pat. Nos. 5,578,832, 5,631,734,5,834,758, 5,936,324, 5,981,956, 6,025,601, 6,141,096, 6,185,030,6,490,533, 6,650,411, 6,643,015 and 6,201,639, in International PatentApplication PCT/US99/06097 (published as WO99/47964), in U.S. patentapplication Ser. No. 09/682,837 (abandoned), and in U.S. ProvisionalPatent Application Ser. Nos. 60/364,731 (expired), 60/396,457 (expired),and 60/435,178 (expired), each of which patent and patent application ishereby incorporated by reference in its entirety for all purposes.

Examples of probe arrays and associated cartridges or housings may befound in U.S. Pat. Nos. 5,945,334, 6,287,850, 6,399,365, and 6,551,817,each of which is also hereby incorporated by reference in its entiretyfor all purposes. In addition, some embodiments of the probe array maybe associated with pegs or posts, where for instance probe array 140 maybe affixed via gluing, welding, or other means known in the related artto the peg or post that may be operatively coupled to a tray, strip orother type of similar substrate. Examples with embodiments of the probearray associated with pegs or posts may be found in U.S. patentapplication Ser. No. 10/826,577 (abandoned).

Labeled targets hybridized to probe arrays may be detected using variousdevices, sometimes referred to as scanners, as described above withrespect to methods and apparatus for signal detection. For example,scanners image the targets by detecting fluorescent or other emissionsfrom labels associated with target molecules, or by detectingtransmitted, reflected, or scattered radiation. A typical scheme employsoptical and other elements to provide excitation light and toselectively collect the emissions.

For example, array scanners provide a signal representing theintensities (and possibly other characteristics, such as color that maybe associated with a detected wavelength) of the detected emissions orreflected wavelengths of light, as well as the locations on thesubstrate where the emissions or reflected wavelengths were detected.Typically, the signal includes intensity information corresponding toelemental sub-areas of the scanned substrate. The term “elemental” inthis context means that the intensities, and/or other characteristics,of the emissions or reflected wavelengths from this area each arerepresented by a single value. When displayed as an image for viewing orprocessing, elemental picture elements, or pixels, often represent thisinformation. Thus, in the present example, a pixel may have a singlevalue representing the intensity of the elemental sub-area of thesubstrate from which the emissions or reflected wavelengths werescanned. The pixel may also have another value representing anothercharacteristic, such as color, positive or negative image, or other typeof image representation. The size of a pixel may vary in differentembodiments and could include a 2.5 μm, 1.5 μm, 1.0 μm, or sub-micronpixel size. Two examples where the signal may be incorporated into dataare data files in the form *.dat or *.tif as generated respectively byAffymetrix Microarray Suite (described in U.S. Pat. No. 7,031,846) basedon images scanned from GeneChip® arrays. Examples of scanner systemsthat may be implemented with embodiments of the invention include U.S.patent application Ser. No. 10/389,194 (allowed), and Ser. No.11/260,617 (allowed), and U.S. Pat. Nos. 7,148,492 and 7,317,415, eachof which are incorporated by reference above.

Examples of autoloaders and probe array storage instruments aredescribed in U.S. patent application Ser. No. 10/389,194 (allowed) andSer. No. 10/684,160 (abandoned); and U.S. Pat. Nos. 6,511,277 and6,604,902 each of which are hereby incorporated by reference in theirentireties for all purposes.

Examples of fluid handling elements and methods for mixing fluids in achamber are provided in U.S. patent application Ser. No. 11/017,095(abandoned), which is hereby incorporated by reference herein in itsentirety for all purposes.

Additional examples of hybridization and other type of probe arrayprocessing instruments are described in U.S. patent application Ser.Nos. 10/684,160 and 10/712,860, both of which are hereby incorporated byreference herein in their entireties for all purposes.

It will be understood by those of ordinary skill in the relevant artthat there are many possible configurations of the components of theaforementioned systems including computers which may be employed in thepresent methods. Processors may be commercially available or it may beone or more different processors that are or will become available. Someembodiments of processors may also include what are referred to asmulti-core processors and/or be enabled to employ parallel processingtechnology in a single or multi-core configuration. System memory may beany of a variety of known or future memory storage devices. Examplesinclude any commonly available random access memory (RAM), magneticmedium such as a resident hard disk or tape, an optical medium such as aread and write compact disc, or other memory storage device. Memorystorage devices may be any of a variety of known or future devices,including a compact disk drive, a tape drive, a removable hard diskdrive, USB or flash drive, or a diskette drive. Such types of memorystorage devices typically read from, and/or write to, a program storagemedium (not shown) such as, respectively, a compact disk, magnetic tape,removable hard disk, USB or flash drive, or floppy diskette. Any ofthese program storage media, or others now in use or that may later bedeveloped, may be considered a computer program product. As will beappreciated, these program storage media typically store a computersoftware program, such as the programs described in more detail below,and/or data. Computer software programs, also called computer controllogic, typically may be stored in system memory and/or the programstorage device used in conjunction with a memory storage device.

In some embodiments, a computer program product is described comprisinga computer usable medium having control logic (computer softwareprogram, including program code) stored therein. The control logic, whenexecuted by a processor, causes the processor to perform functionsdescribed herein. In other embodiments, some functions are implementedprimarily in hardware using, for example, a hardware state machine.Implementation of the hardware state machine so as to perform thefunctions described herein will be apparent to those skilled in therelevant arts.

Input-output controllers could include any of a variety of known devicesfor accepting and processing information from a user, whether a human ora machine, whether local or remote. Such devices include, for example,modem cards, wireless cards, network interface cards, sound cards, orother types of controllers for any of a variety of known input devices.Output controllers of input-output controllers could include controllersfor any of a variety of known display devices for presenting informationto a user, whether a human or a machine, whether local or remote.Functional elements of a computer communicate with each other via asystem bus. Some of these communications may be accomplished inalternative embodiments using network or other types of remotecommunications.

As will be evident to those skilled in the relevant art, an instrumentcontrol and image processing application, such as for instance animplementation of instrument control and image processing applications,if implemented in software, may be loaded into and executed from systemmemory and/or a memory storage device. All or portions of the instrumentcontrol and image processing applications may also reside in a read-onlymemory or similar device of memory storage device, such devices notrequiring that the instrument control and image processing applicationsfirst be loaded through input-output controllers. It will be understoodby those skilled in the relevant art that the instrument control andimage processing applications, or portions of it, may be loaded by aprocessor in a known manner into system memory, or cache memory, orboth, as advantageous for execution. Library files, experiment data, andinternet client may be stored in system memory. For example, experimentdata could include data related to one or more experiments or assayssuch as excitation wavelength ranges, emission wavelength ranges,extinction coefficients and/or associated excitation power level values,or other values associated with one or more fluorescent labels. One ofskill in the art is aware that there are various gene or genomeannotation sites on the internet which may be accessible using software,such as the application described above, and whose information may bedisplayed by the software for the user's consideration.

Instrument control and image processing applications may comprise any ofa variety of known or future image processing applications. Someexamples of known instrument control and image processing applicationsinclude the Affymetrix Microarray Suite, and Affymetrix GENECHIP®Operating Software (hereafter referred to as GCOS) applications.Typically, embodiments of applications may be loaded into system memoryand/or a memory storage device through one of any number of availableinput devices.

The presently disclosed applications may, in the present implementation,provide one or more interactive graphical user interfaces (GUI) thatallows a user to make selections based upon information presented in anembodiment of a GUI. Those of ordinary skill will recognize that GUIembodiments may be coded in various language formats such as an HTML,XHTML, XML, javascript, Jscript, or other language known to those ofordinary skill in the art used for the creation of enhancement ofviewable Web Pages.

Examples of instrument control via a GUI or other interface is providedin U.S. patent application Ser. No. 10/764,663 (abandoned), which ishereby incorporated by reference herein in its entirety for allpurposes.

IV. Specific Embodiments—Data Analysis and Application of CovariateAdjusters

Disclosed herein are methods, systems, software and related articlespertaining to software applications and algorithms available thereinwhich aid in the identification, analysis, manipulation and display oflarge amounts of complex genetic data. Detection in a genetic microarrayexperiment, by scanning of the intensities of fluorescent signals on aprobe array, typically generates a data file containing signal intensitydata, referred to as a “.CEL” file which is saved on a computer whichruns the system performing the scanning. The .CEL files may be convertedinto .cnchp or .cychp files (types of .chp files) which representnormalized intensity values obtained from the probe array during ahybridization experiment. The values are normalized by obtaining a ratioof the value with respect to a baseline reference set of normal samples.Other algorithmic manipulations may be applied to further normalize andcorrect the data for spurious artifacts.

An exemplary experimental flow of the data analysis for the disclosedsystem and process may proceed as follows. First, intensity values maybe obtained from the probe array by a scanner connected within thesystem. The intensity values may then be collected in the memory deviceof a controlling computer in a .dat file (or multiple such files). Then,the .dat file may be converted to a .CEL file as explained above. Third,an HMM algorithm, or similar mathematical analysis algorithm, may beemployed by software to convert the data into normalized data, exportedas a .cychp file, or a .cnchp file, for instance. These .cychp and/or.cnchp data files contain information utilized by application browsersto display the information in multiple colors and a variety of windows,showing various segments of genomic sequence information to the user forvisual inspection and analysis.

The display of the data may be in color and may be interactive, allowingthe user to define various functionalities and various segments ofgenome being investigated by the experiment. For instance, theapplication may contain programming that allows the display of a map ofthe entire genome of the animal, bacteria, plant or other entity ofinterest. The genome may be human, mouse, insect, plant, bacterial orany other type of genome. The genetic map displayed to the user may bein color and particularly may make use of various colors to signifydifferent functionalities or characteristics of the genetic data.Furthermore, the application may enable a user to interface directlywith the genomic map obtained from the data, which may display, forinstance, the identities of the various SNP sequences identified by thegenetic experiment(s). The genetic map may be depicted in the form ofchromosomes, or shapes which mimic or reflect traditional depictions ofchromosome shapes as seen by, for instance, a cytogenetecist examiningchromosomes through a microscope. See, for instance, U.S. PatentApplication Publication Number 2011/02578, incorporated herein byreference for all purposes.

Various other concepts of organization and presentation of SNP data,copy number calls and segmentation analysis may be found in U.S. patentapplication Ser. No. 12/986,986, filed on Jan. 7, 2011 and entitled“Differential Filtering of Genetic Data,” the entirety of which isincorporated herein by reference for all purposes. The ×986 applicationprovides much greater detail on various embodiments of acquisition ofsuch data and the display of such data for user analysis andmanipulation. The ×986 application provides various exemplaryembodiments which can be used to display the data after manipulation byapplication of the various algorithms and methods disclosed herein. Thatis, the present methods and algorithms provide means by which microarrayexperimental data may be normalized, corrected, analyzed and generallymade more consistent from experiment to experiment across arrays, morerobust and more dependable, by removal of various independent variablesand by better display of the data. Upon execution of these methods, thedata upon which the present methods operate may be displayed in variousforms and formats by, for instance, the software applications disclosedin the ×986 application.

The Affymetrix Chromosome Analysis Suite (ChAS, available fromAffymetrix, Inc., Santa Clara, Calif.) is a non-limiting example of suchcomputer programs or software. The various possible features that may bepresented in the software program display are disclosed in exemplaryembodiments in the ChAS software. The ChAS User Manual, which isavailable and freely downloadable in .pdf form from the Affymetrixwebsite, is specifically incorporated herein by reference in itsentirety for all purposes.

The intensity values detected emanating from the microarray areempirically determined by a scanning laser which detects signals from,for instance, the hybridization of a labeled genetic sample to the probearray, as explained above. These intensity values may be compared insilico to the values in a reference, which may be a reference model,e.g. an ideal model of the predicted or expected or average outcome ofthe experiment, or it may be another .CEL data file or .rmf file. Thiscomparison is made by determining a ratio of the experimentallydetermined intensity values in a .CEL file with the reference. Such aratio is generated in silico by the software application as a log₂ratio.

Thus, the log₂ ratio is the normalized intensity data obtained from theprobe array. This ratio also represents the copy number of theexperimental genetic material represented by the probes on a probe byprobe localization basis. In other words, the ratio value generated bythe software application represents the number of copies of a given SNPor other non-polymorphic marker, detected within the experimental genesample, with respect to the reference. In this manner, the user mayobtain what is commonly referred to as the copy number of a gene or SNPor marker. The log₂ ratio is most commonly the log₂ of the ratio ofmeasured signal intensity value (for any given marker) divided by medianreference signal intensity value.

In general, the present specific embodiments are related to themanipulation of intensity value data so as to achieve more robust andhigher quality data which is more reproducible in the field ofmicroarray assays in general. For instance, covariate adjustors may beemployed to make corrections in array data based on orthogonal knowncharacteristics. Types of covariate adjusters which a user maygenerically apply to a given set of intensity value data include, butare not limited to, adjusters for GC content (Super OC covariateadjuster), etc. A user of the type of software program, such as ChASdescribed above, could manually select one or more different kinds ofcovariate adjusters to apply to a particular data set, depending on theneeds and the character of the data produced by the particularexperiment.

Additionally, high pass filters may be employed for signal processing tomanipulate image data obtained from the microarray experiment. The imageused in this high pass filter may be, for instance, a pseudo image whichis generated using a log₂ ratio value of intensities rather than simplyusing and/or displaying to the user the raw image data obtained from DATor CEL files. Transforming the image data by the log₂ ratio uncoversimportant gradients in the images that can only be seen with this methodand therefore can only then be corrected using this method.

Finally, the data may also be manipulated and improved upon throughsignal restoration algorithms. Such signal restoration algorithms mayemployee Bayes wavelet shrinkage methods used on log₂ ratio data.

The following is a high level overview of how copy number (CN) calls aregenerated within software. The copy number workflow starts with theintensities on the array, include normalization and scaling, referenceset ratios, log₂ transformation, CN state segmentation, and how CNsegment calls are made. (See, FIG. 1).

Microarrays are scanned and processed by scanners available in the artand programs run by computers in the scanning system. Commonly used andavailable software, such as AGCC scanner software package available fromAffymetrix, Inc., Santa Clara, Calif., aligns a grid on the .DAT file(the original scanned image) to identify each microarray feature andcalculates the signal from each feature. This process uses the .DATfile, containing the raw signal, and creates a .CEL file, which containsa single signal intensity for each feature. The .CEL file is used forall downstream analyses.

Beginning with the raw signal data in the .CEL file, a series of stepsmay be implemented that perform probe set summarization, normalization,removal of variation caused by known properties and residual variation,and completing with calling genotypes, copy number segments and LOHsegments. The complete detail of exemplary steps performed in theexemplary workflow is shown in FIG. 1 and each step is briefly describedin the subsequent text. The first level of covariate adjustors operateson the raw signal.

Signal Level Covariate Adjusters

The Fragment Adapter Covariate Adjustor: After restriction digest of thegenomic sample, an restriction enzyme-specific adaptor may be ligatedonto the cohesive end termini. For example, a restriction enzyme such asNsp I, commonly employed in sample processing for microarrayexperiments, may be added to the sample genomic material to break downthe long strands of DNA into shorter strands that are easier tomanipulate for the purposes of performing the microarray experiment.Since Nsp I is a 6-nucleotide cutter with degenerate sites, meaning thatthey contain one or more base pairs that are not specifically defined,these ends are of various sequences and the ligated adaptors are avariety of sequences. The exact sequences of the cut site and ligationadaptor have an effect on the overall efficiency of ligation andsubsequent PCR amplification. The Adaptor Covariate Adjustor is able tocorrect for these differences by normalizing the signals for eachadaptor/cut site sequence class to an overall median.

Fragment Length Covariate Adjustor: The length of each fragment impactsthe efficiency of PCR amplification and therefore the signal. Fragmentsof 300-500 bp are amplified with the highest efficiency and the degreeof amplification tapers off as the fragments get longer. The LengthCovariate Adjustor corrects for these differences by normalizing thesignals for a series of fragment size bins to an overall median.

Dual Quantile Normalization

Dual quantile normalization is a two-phase process where probes used forcopy number detection and probes used for SNP genotype detection arenormalized separately. In both cases, a normalization sketch is builtusing the autosomal probes in the reference set. The normalizationsketch is the prototype distribution of probe intensities that defineswhat this distribution looks like for all arrays. The single sampleautosomal probes are fit to the sketch and the X and Y probes areinterpolated into the distribution.

Quantile normalization makes the assumption that the distribution ofprobes on the array is fairly consistent from array to array. Since theX-chromosome is one of the largest chromosomes (155 Mbp, ˜5% of thegenome), differences between males and females would stretch thisassumption. That is why the quantile normalization focuses on creatingan autosomal sketch and normalizing the autosome to it. The X and Ychromosome probes are then handled in a special way. Each of them ismatched to the closest pre-normalization signal value. Based on thatmatch, their normalized signal should be close to the signal for thevery same autosomal probe. So the normalized values for X and Y probesare simply “looked-up” in the pre-normalization autosomal sketch, andtransformed to the post-normalization value.

Copy Number Work Flow

A general exemplary workflow for processing raw image intensity dataobtained from a microarray experiment may be as follows. The followingexplanation makes reference to the left side of the flowchart of FIG. 1,which pertains to the various steps involved in manipulation of signalintensity data for the purpose of determining copy number (CN).

Log₂ ratios for each marker are calculated relative to the referencesignal profile. The log 2 ratio is simplylog₂(sample_(m))−log₂(reference_(m)), for each marker, “m”.

After the log₂ ratio calculation is made, optionally a high pass filterimage correction step may be employed (explained in more detail below).Since most probes map to genomic markers associated with a normal copynumber, most log₂ ratios should be centered at a value of zero. Also,since markers from any genomic region are scattered across the surfaceof the microarray, regions of altered copy number will not appear asregional changes on the microarray image.

Some samples do reveal spatial trends away from zero that are gradualand this spatial bias when scattered back across the genome exhibitsitself as added noise in the log 2 ratios. The High Pass Filter ImageCorrection identifies these gradual spatial trends and adjusts log 2ratios to remove the spatial bias and lower the level of noise.

Log 2 Ratio-Level Covariate Adjustors

The Super GC Covariate Adjuster

It is well known that the nucleotides G and C hybridize with each othermuch stronger than A and T. Thus, when a fragment of DNA contains adisproportionate amount of the nucleotides G and C, and this fragment iseither used as a probe on a microarray or is part of a labeled fragmentto be hybridized to a microarray, spurious events can occur in manysteps of the experiment. For instance, the GC content of genomic DNAsequences impacts probe signal dose-response and therefore probe log₂ratios. In other words, when the sample is amplified it takes less GClabeled fragments to hybridize to the microarray and provide a goodsignal than fragments containing more A and T nucleotides, due to thedifferences in hybridization strength. This may make it appear as thoughthere are more fragments containing the over-abundance of G and Cnucleotides than other fragments, and lead to an artifact in thedetermination of copy number.

In addition, the genomic GC content of the fragments and the 500 kbpsurrounding the probe (local GC) impacts the efficiency of targetpreparation in the genomic region of each probe. It is well known thatwhen PCR is performed to amplify the genetic sample material, polymeraseenzymes tend to have differences in enzymatic efficiency when amplifyingGC rich segments of DNA as compared to AT rich segments of DNA. Thus,polymerase enzymes may artificially over-amplify AT-rich fragments ofDNA as compared to GC-rich fragments. The subsequent labeling of thesefragments and hybridization to a micrarray can again lead to spuriousresults, i.e. an artificially enhanced signal for AT-rich sequences overGC-rich sequences in the genomic sample.

Each one of these independent variables can be individually controlledby separate covariate adjusters. A specific algorithmic adjuster may beapplied to adjust for local GC (GC content/concentration within aspecific window of length across the probe or fragment), fragment GC andprobe GC. However, in doing so, one would find that upon adjustment ofthe intensity data for one variable, one of the other two variableswould need re-adjustment, and so on. In other words, because all threevariables concern GC content in the sample or probes, all three areinterdependent. A solution to this problem presents itself in the formof a complex algorithmic calculation wherein all three variables aresimultaneously adjusted by one “master” covariate adjuster. This is whatthe present invention accomplishes. Though it seems like perhaps asimple solution, it is not simple. An algorithm must be designed,operating in three-dimensional space, which simultaneously analyzes allthe data produced by a scan of an entire microarray, and adjust formultiple variables to arrive at a single, well-defined, well-controlledsolution which eliminates much of the spurious and artifactualvariability introduced by such independent variables. We hereinafterrefer to this “master” covariate adjuster for GC content as the “SuperGC Covariate Adjuster.”

The Super GC Covariate Adjustor combines the probe GC content, thefragment GC content and the local GC content into one covariate thatcorrects for log 2 ratio differences based on the combination of GCcontents associated with each probe.

Reference Intensity Covariate Adjustor

As explained above, covariate adjusters in the present applicationoperate by first placing various intensity values into bins. Artificial,numerical bins are constructed comprising specific ranges of intensityvalues. All intensity values from a particular microarray experiment areplaced into their respective bin. Then, all values in those bins aresimultaneously adjusted in the same direction and in the same quantity,so they are all treated identically to remove and account for theindependent variable introduced into the data. Of course, if one were toconstruct bins such that there is a single bin for every value, everysingle value in the experiment would be individually adjusted oradjustable. This would lead to nonsensical results since no value wouldbe true to the experiment as quantified by the scanner. However, too fewbins into which wide ranges of values are placed would lead tounder-correction of the data and less than desirable results. Thus,choosing the appropriate number of bins for each microarray experimentis required.

Probes in different intensity categories have different dose responsesin log 2 ratio space. Using Reference Set probes to define bins based onprobe intensity, the single sample probes are binned and the median ofthe distribution of log₂ ratios within each bin is adjusted to themedian log 2 ratio of the corresponding bin from the reference set.

Marker Type Covariate Adjustor

Polymorphic probes mounted onto microarrays may be designed for SNPdetection and other, non-polymorphic, probes may be designed for copynumber detection. These two types of probes have different propertiesand different dose responses, i.e. they produce different levels ofsignal depending on the quantity of labeled sample present when it ishybridized to the microarray. The Marker Type Covariate Adjustornormalizes the median log₂ ratios of SNP and CN markers to account fordifferences in log₂ ratios between these two kinds of probes.

Median Autosome Normalization

This final level of normalization simply shifts the median log 2 ratioof the autosomes to a copy-number state equal to 2, i.e. a log₂ ratio of0, which represents a “normal” sample since in most organisms studiedthus far, it has been consistently found that each organisms has exactlytwo copies of every gene, unless there has been an event which adverselyimpacts the chromosome or genome to either amplify or reduce that normalcopy number of 2.

Systematic Residual Variability Removal

Even after all of the Covariate Adjustors are applied to the data, thereare some residual variations with unknown origins. The SystematicResidual Variability Removal step matches sample variability to theresidual variability of the reference set, and when matched, correctsthe data to remove the residual variable that was matched, i.e. if thevariable consistently appears in both the reference set and theexperimental data, it can safely be assumed that the variable arose dueto some independent factor impacting every experiment from the system.

Remaining Steps in FIG. 1

The Signal Restoration step, as outlined in the left side of the flowdiagram of FIG. 1, is explained in more detail below. Segmentation,Smoothing/Joining and Visualization are disclosed in, for instance, U.S.patent application Ser. No. 12/986,986, filed on Jan. 7, 2011 andentitled “Differential Filtering of Genetic Data,” the entirety of whichis incorporated herein by reference for all purposes. While the rightside of FIG. 1 is not expressly discussed herein, it is to be understoodthat other covariate adjusters may be developed and applied to thedetermination of SNP sequence in the same manner as explained above forCopy Number variations.

Within the exemplary sample analysis and data gathering work flowpresented above, methods and systems are disclosed herein for usingcovariate information to adjust signal and log₂ ratio information in theanalysis of intensity values obtained from microarray experiments. Themethodology proposed is intended to be generic so that any covariate maybe used in this framework. There may be numerous covariate adjustersthat are associated with each probe/marker on a microarray, and forevery step in the sample preparation process, and for every flaw or biasthat may be found in the system used to detect the signals—such as thescanner or the chip or the scanning arm, for instance. Some of thesecovariate adjusters can reasonably be assumed to not be directlyassociated with biologically meaningful copy number differences.Furthermore, it is possible to observe that between different runs ofthe same sample, variation in the signal (or log₂ response) as afunction of these covariates differs. Various methods for removingcovariate related differences are provided herein.

Covariate adjusters may be applied in many methodologies to account andcorrect for many different types of independent variables, such as, butnot limited to: fragment adaptor type, fragment length, fragment GCcontent, probe GC content, and local (regional) GC content.

Development and Application of Covariate Adjusters to Microarray Data

Disclosed are methodologies for adjusting signal information relative toa covariate. To begin with, signal intensities (which may be, forinstance, reflectance off of the array and/or to fluorescence emanatingfrom labeled sample hybridized to the array) may be binned based ontheir values as discussed above. Binning of values is a method ofsorting such experimental values or measured quantities based on a valuerange.

One way to examine data stratified this way is using boxplots as shownin FIG. 2. FIG. 2 represents the binning of fragment length. Asexplained above, in a standard microarray experiment, the geneticmaterial must first be prepared by digestion (fragmentation, typicallyusing a restriction enzyme, such as NspI) and then amplification,followed typically by further digestion with Dnase and then labeling.Thus, in the preparation of genetic samples, many different fragmentlengths of sample may be generated, as reflected in FIG. 2. The fragmentlengths portrayed in this figure and later figures relates to the lengthof the fragments of DNA that result from the restriction enzymedigestion step of the non-amplified genetic material. However, one coulddevelop a covariate adjuster which also may address any bias inherent inthe length of amplified fragments of DNA resulting from Dnase digestion.Furthermore, it has been found that fragment length can have adetrimental, or favorable, impact on the hybridization of that labeledsample fragment to the microarray. For instance, if the fragment is toolong, it may not hybridize well to the microarray-bound probe due tosteric issues or other physical strain or stress present during theassay. Likewise, shorter fragments may have trouble hybridizing to theirrespective probes on the microarray because there may not be enoughnucleotides to form a stable hybridization complex that can withstandthe washing and other steps which form part of the assay. Each bar onthe graph in FIG. 2 represents the measured intensity values for samplefragments which fall within the fragment length range defined in basepairs (bp) by the boxes on the x-axis of the box plot. The center ofeach box summarizes the relationship between signal and the covariate.The values being binned in FIG. 2 are fragment length.

The signal may also be summarized irrespective of the level of thecovariate by using the overall median of the intensity values asdepicted in FIG. 3. The approach reflected in FIG. 3 is to adjust themedian of each box to fall at the level of the overall median. A scalingtransformation may be used to achieve this result as visually depictedin FIG. 4.

Specifically, the scaling factor for each covariate bin median is givenby the ratio of the overall median to the median for that bin. So forbin the scaling factor is M/M_(i) where M is the overall median andM_(i) is the median for that bin, for instance as visually depicted inFIG. 5. For each bin shown in FIG. 2, all signal intensities are thenscaled using the same computed scale factor (covariate adjustment) toarrive at the result visually depicted in FIG. 6. The flow diagram ofFIG. 7 provides an example of how the algorithmic analysis may proceed.

It is noted that chromosome X and Y probe signals are not used todetermine the scaling factors in human samples, though they may berescaled based on which covariate bin they fall in. Additionally, it isnoted that in a typical use case, different marker classes should beadjusted separately (here CN and SNP are the primary classes expected).Thus, one would bin SNP marker probes in a separate covariate adjustmentanalysis and a CN probe in another, separate, covariate adjustmentanalysis since each of these probe types experience differentindependent variables which impact the performance of these probes.Further, multiple covariate values may be adjusted, one by one, insequence or in parallel, in whatever order is most appropriate for thespecific use intended, Software may be designed such that the user mayindicate which covariate adjusters to apply and in what order andquantity, depending on the type of experiment being performed, the typeof equipment or system or system components being used, type ofmicroarray used and the like.

The methodology proposed for adjusting log₂ ratios, or for any of thecovariate adjusters mentioned in this application, may also basicallyfollow the generic analysis scheme discussed above and depicted in FIGS.2-7. More specifically, log₂ ratios may be binned by their covariatevalues, such as is shown in the boxplots of FIG. 8. FIG. 8 depicts apre-adjustment (before application of covariate) plot of log 2 ratio forbinned signal intensities, binned by percent GC content. The x-axisplots GC content in percentages (% per oligonucleotide fragment) and they axis plots signal intensity in relative intensity units. The lineacross the middle represents the target overall median intensity.

Each bin may be summarized based on the median log₂ ratio, as shown inFIG. 9. To eliminate differences the goal of the present methods is toshift every median towards 0, i.e. no change, as depicted in FIG. 10. Toeliminate differences the goal is to shift every median towards 0, i.e.no change, which is accomplished by subtracting the median value foreach. Each log 2 ratio is adjusted by subtracting the median value thatcorresponds to the covariate bin within which it falls. The final resultis depicted in FIG. 11.

The flow chart of FIG. 12 provides an exemplary outline of the steps ofthe algorithm-based analysis as they may be employed in the presentlydisclosed methods and systems. It should be noted that chromosome X andY may be excluded when determining the adjustment medians for each bin,though they are adjusted. Additionally, for some covariates it may makesense to adjust both location (via the median) and scale. This secondadjustment is done on the inter-quartile range (IQR) by multiplying by arelevant scale factor. The IQR is the difference between the 75^(th)percentile and 25^(th) percentile. Additionally, as mentioned above,different marker classes could be adjusted separately (here CN and SNPare the primary classes expected). Further, multiple covariate valuescan be adjusted, one by one, in sequence, or in parallel, whicheverappears most appropriate or relevant for the specific application.

Binning Covariates

The disclosed algorithms is that covariates are to be treated asdiscrete variables. Covariates that are continuous are turned intodiscrete variables by means of binning. There are multiple ways in whichthis can be accomplished. For example, one may employ equal spacing,which means that the spaces between the cut points are set as equal inmeasurements of the covariate. (See, FIG. 13). Depending on thedistribution of the covariate across the available marker set, this canlead to wildly different numbers of probes in each bin. This also meansthat low frequency bins could yield radical results because they arebased on just a small number of data points, i.e. markers. Anotherexample is to use bins where each bin is defined such that each has thesame number of probes or data points therein. This will result in binswhich are uneven in the spacing. FIG. 13 illustrates these two exemplaryalternatives for binning of a covariate.

These two exemplary algorithms may be placed in the processing pipelinefor both reference building and single sample analysis. The ideal placeto have the signal adjustment stage is before the dual quantilenormalization. The preferred embodiment provides for the log 2 ratioadjustment to occur after computation of the log₂ ratios and before anysubsequent adjustments. In reference mode, the goal is that waves aredetermined around residual unexplained variability. Similarly, the wavecorrection (“estimation of wave correction”) in single sample analysis(SSA) mode is applied after all covariate based signal and log₂ ratioadjustment has occurred (see FIGS. 14 and 15).

These methods will be able to handle any ordering of covariates, and theorder in which they are to be used can be specified by the user.Further, the usage of the same covariate for both signal and log₂ ratiocorrection stages is possible and methods employing the same covariatemultiple times with different settings at the same adjustment step arealso possible. The sequential nature of application of the variouscovariate adjusters disclosed herein assumes that there is nointeraction effect among potential covariates. However, if such aninteraction does exist, the appropriate way to deal with it in thisframework is to create a new discrete covariate to adjust bothcovariates simultaneously. Markers for which information is missing fora given covariate could be tagged to undergo no correction. Covariateinformation is typically, but not necessarily, stored in the annotationfile associated with the particular data set being analyzed.

An exemplary implementation of a log₂ ratio algorithm generic adjusteris as follows below. Though the code below is drafted in a specificcomputer language, the same steps may be implemented in other computerlanguages known in the art:

## LR - log ratio ## covariate - covariate ## markerclass - categoricalvariable. ## continuous - TRUE/FALSE ## nbins - Bin variable into nomore than this number of bins (has no effect if variable is notcontinuous) Adjuster <− function(LR, chromosome, covariate, markerclass,continuous=TRUE, nbins=25, even.quantile=FALSE,standarize.spread=FALSE){bin.function <− function(covariate, continuous=TRUE, nbins=25,even.quantile=FALSE){ if (!continuous){ ## do not do any re-binning if(is.factor(covariate)){ return(covariate) } else {return(as.factor(covariate)) } } else { if (!even.quantile){tentative.cutting <− cut(covariate,nbins) } else { tentative.cutting <−cut(covariate,quantile(covariate,seq(0,1,length=nbins+1)),include.lowest=TRUE)} return(tentative.cutting) } } for (cur.type in unique(markerclass)){ #determine the adjustment using autosomal markers only cur.LR <−LR[markerclass==cur.type] cur.covariate <−covariate[markerclass==cur.type] cur.chromosome <−chromosome[markerclass==cur.type] cur.bins <−bin.function(cur.covariate, continuous, nbins, even.quantile) adj <−lapply(split(cur.LR[is.element(cur.chromosome,1:22)],cur.bins[is.element(cur.chromosome, 1:22)]),median) if(standarize.spread){ adj.IQR <−lapply(split(cur.LR[is.element(cur.chromosome,1:22)],cur.bins[is.element(cur.chromosome, 1:22)]),IQR) med.IQR <−median(do.call(“c”,adj.IQR)) for (i in 1:length(adj.IQR)){ adj.IQR[[i]]<− med.IQR/adj.IQR[[i]] } } # apply it to all markers irrespective ofchromosome for (bin in levels(cur.bins)){ cur.adj <− adj[[bin]] if(!standarize.spread){ cur.LRfcur.bins == bin] <− cur.LR[cur.bins == bin]− cur.adj } else { cur.LR[cur.bins = bin] <− (cur.LR[cur.bins == bin] −cur.adj)*adj.IQR[[bin]] }  } # now reassign the corrected value backLR[markerclass==cur.type] <− cur.LR }  LR }

An exemplary implementation of a signal adjuster algorithm is asfollows:

## Signal - Probe signal ## covariate - covariate ## markerclass -categorical variable. ## continuous - TRUE/FALSE classification ofcovariate variable ## nbins - Bin variable into no more than this numberof bins (has no effect if variable is not continuous) ## even.quantile -should binning be done by splitting covariate Adjuster <−function(Signal, chromosome, covariate, markerclass, continuous=TRUE,nbins=25, even.quantile=FALSE){ bin.function <− function(covariate,continuous=TRUE, nbins=25, even.quantile=FALSE){ if (!continuous){ ## donot do any re-binning return(as.factor(covariate)) } else { if(!even.quantile){ tentative.cutting <− cut(covariate,nbins) } else {tentative.cutting <−cut(covariate,quantile(covariate,seq(0,1,length=bins+1)),include.lowest=TRUE)} return(tentative.cutting) } } for (cur.type in unique(markerclass)){cur.signal <− Signal[markerclass==cur.type] cur.covariate <−covariate[markerclass=cur.type] cur.chromosome <−chromosome[markerclass==cur.type] cur.bins <−bin.function(cur.covariate, continuous, nbins, even.quantile) ## Baseadjustment on autosomal markers ## We preserve the current medianintensity in this transformation cur.target <−median(cur.signal[is.element(cur.chromosome,1:22)]) adj.median <−lapply(split(cur.signal[is.element(cur.chromosome, 1:22)],cur.bins[is.element(cur.chromosome, 1:22)]),median) ## apply it to allmarkers irrespective of chromosome for (bin in levels(cur.bins)) {cur.adj <− cur.target/adj.median[[bin]] cur.signal[cur.bins == bin] <−cur.signal[cur.bins == bin]*cur.adj } ## now reassign the correctedvalue back Signal[markerclass==cur.type] <− cur.signal } Signal }

There are multiple types of generic covariate adjusters that may beemployed in the present methods and systems. Furthermore, these genericcovariates (GC) can be employed in different ways, i.e. at differentpoints during data analysis in a microarray experiment. Three basicexemplary generic covariate adjusters include, but are not limited to,fragment GC, local GC and probe GC. All of these covariate adjustersadjust for the percent of the sequence that is either of the twonucleotides G and C. As discussed above, an above-average amount of Gand C nucleotides can impact how that sequence behaves in a standardhybridization-based microarray assay. Fragment GC is employed to correctfor variables associated with sample preparation, i.e. fragmentation ofthe original genetic sample by use of various restriction enzymes, whichmay have a bias in the manner in which they cleave large geneticsamples. Local GC adjusters address the variability observed in thegenetic content, or sequence, of the sample, i.e. the amount or percentof G and C nucleotides found within a specified window of base pairswithin the fragment, centered about a genetic marker (SNP or othercharacteristic sequence) being measured. The window of base pairs beingconsidered for covariant correction may be any window size chosen by theuser. A standard window size for CN and/or SNP analysis would be, forinstance, 500 base pairs. However, the window may be as small as 50,100, 150, 200, 250, 300, 350, 400 or 450 base pairs, or even as large as550, 600, 650, 700, 750, 800, 850, 900, 950 or even 1000 base pairs (1kb). In some cases the window chosen for the local GC covariate adjustermay be even larger, for instance 1.5 kb, 2 kb, 2.5 kb, 3 kb, 3.5 b, 4kb, 4.5 kb or even 5 kb. The local GC covariate length window may evenbe 50 kb, 100 kb, 150 kb, 200 kb, 300 kb, 400 kb, 500 kb, 600 kb, 700kb, 800 kb, 900 kb, 1 mb, 2 mb, 3 mb or longer. The probe GC covariateadjuster adjusts for variables associated with the percent of G and Cnuclcotides contained in the sequence of the probe mounted to themicroarray, to which the genetic sample hybridizes.

These three exemplary GCs interact with each other in ways that alsointroduce further bias. The three covariate adjusters and theircovariates are interrelated in that adjustment of the data set for onecovariate will sometimes require further adjustment of the othercovariates. That is, if a user intends to apply multiple covariateadjusters on one set of data, the problem is that each individualcovariate adjustment may require the data set to be again adjusted bythe other covariate adjusters the user intends to employ. This iterativeprocess could go on indefinitely until a final solution is found. Toaddress this cyclical problem, one can combine all of the neededcovariate adjusters into one “Super GC” covariate adjuster thatsimultaneously adjusts the data to eliminate all biases and independentvariables in one step. This multi-parallel process will thus eliminatethe need to keep doing iterative adjustments, making data analysis muchmore efficient and less resource intensive.

The Super GC covariate is a discretization of multiple covariate space.For instance, the fragment GC-local GC-probe GC space may be combined asreflected in FIG. 16, where every marker has its own fragment GC, localGC and probe GC and can be represented as a point in thethree-dimensional space.

The goal of the discretization is to collect markers of similarcovariates together, and then adjust these markers altogether andsimultaneously as a group. Of course there are many methods by whichthis partitioning of the space could be implemented, with each methodhaving its own advantages and disadvantages. For instance, one methodfor implementing this partitioning of space is to place more emphasis onoptimizing the partitioning in the local GC and fragment GC dimensions,though ultimately all three dimensions are partitioned. Additionally thepartitioning may be performed in a linear manner (by using perpendicularplanes), rather than using non-flat surfaces. SNP markers and CN markersmay be partitioned separately.

One exemplary procedure for achieving this partitioning is as follows.First, considering just the two dimensions of fragment GC and local GC,a regression line may be fit predicting fragment GC as a function oflocal GC, as depicted in FIG. 17. The fitted regression line may then beused to perform a transformation of the indicated space. In particularthe intercept estimate may be subtracted from the fragment GC for eachmarker, and then the axes may be rotated such that the regression linebecomes the new horizontal axes. The rotation may be performed in polarcoordinate space, yielding a new coordinate system (see FIG. 18).

The second step generally would be to partition the space, as in FIG.19. The space may be divided into equally populated bins by using linearseparators (perpendicular to horizontal axis). For instance, the binningmay be into five equal portions (each containing 20% of the data). Then,each of those bins is further divided, but this time in the seconddimension. Again, each bin is designed to be equally populated, withfive bins. A further round of partitioning is carried out in the firstdimension. Finally, there may be one more partitioning in the seconddimension (not shown). Ultimately this exemplary method of partitioning,or binning, yields 625 distinct groupings in the transformed space (andthus also in the fragment GC/local GC space).

FIG. 20 shows the 625 distinct bins in the exemplary fragment GC andlocal GC space. For each bin, one round of partitioning is carried outin the Probe GC dimension (also five bins). This results in 3125distinctly unique bins. An index value is assigned to each bin and thisbecomes the discrete covariate known as the Super GC.

Various other covariate adjusters may be employed in a similar matter asexemplified by the Super GC discussed above. However, some assumptionsmust usually be made when employing covariate adjusters to analyzemicroarray data. For instance, common assumptions may be one or more ofthe following non-limiting examples: (1) assume that the covariate doesnot correlate directly with real, biologically meaningful copy numberchanges in the genetic sample being tested, (2) assume that covariatebehavior on chromosomes X and Y is not materially different than on theautosomes, and (3) assume that there is no interaction betweencovariates, i.e. if the data is adjusted by covariate A and then by B,the effect of A remains removed.

Another category of covariate adjusters is annotation-based covariates.There are a number of different annotation-based covariates that couldbe employed in the presently disclosed systems and methods. FIG. 21depicts an example of the values of the covariates, broken down by CN orSNP marker and also by chromosome. Distinct differences may be visiblein microarray genetic experiment data, between the CN and SNP markers,for fragment length and other covariates, as shown. Thus, in oneembodiment, separate covariate adjustments may be made for CNdetermination and SNP detection. Furthermore, it is observed that thereis not much evidence of differences between chromosome X and Y andautosomes.

Pair-wise comparison of some of the covariates demonstrate that there iscorrelational structure, particularly in the case of the various flavorsof GC (see, for example, FIGS. 26-29).

Another annotation based covariate that could be explored is the markertype, i.e. describing whether a marker is a copy number(non-polymorphic) or SNP (polymorphic) marker.

Signal Based Covariates

The intent of employing covariate adjusters is to remove technicalvariability in signal intensity values and variability in the log₂ ratiovalues which may vary on a sample-by-sample basis. These differences insignal intensity values would be directly related to real copy numberchange, which is one of the assumptions of the presently disclosedcovariate adjuster methodology. Therefore, it may appear illogical toemploy a covariate based on a signal measurement. However, there is onesignal quantity that is assumed to be, and mostly constructed by designto be, unrelated to copy number change. This is the median referenceintensity value stored in a standard reference file which typicallyaccompanies microarray experimental data files in .CEL format, etc.Using this as a covariate allows for the correction of intensity biasesthat may not be addressed by other covariates.

Due to the way content is typically selected for microarrays, markerintensities on chromosome X may be dimmer than those for the autosomes.There may also be differences for chromosome Y marker intensities ascompared to autosomal marker intensities. Because of these differences,it may not be useful to apply the median reference intensity covariateto markers on chromosomes X and Y.

Using R, a pipeline may be built that incorporates these types ofcovariate adjusters discussed above. As depicted in FIG. 30 and FIG. 31,included may optionally be both reference building mode and singlesample analysis mode. However, such a pipeline does not necessarily haveto have wave correction implementations.

A large number of combinations of different covariate adjustments may beexplored utilizing the presently disclosed systems and methods. Areference file may be generated for each explored combination, and thena series of experimental samples may be processed via SSA.

To simplify testing, the parametrization for each covariate adjuster maybe determined beforehand. Covariates may optionally be tested as bothsignal and log₂ ratio corrections to determine which provides the mostoptimal results in terms of the final adjusted data.

Signal Restoration

Signal restoration is an application of Bayes wavelet shrinkage to thelog₂ ratio values (described above) associated with genetic markersfound on probes and/or genetic sample fragments. For instance, see“Multivariate Bayes Wavelet Shrinkage and Applications,” by GabrielHuerta, J. Applied Statistics, 32(5):529-542, 2005 (incorporated hereinby reference in its entirety for all purposes). The term waveletshrinkage refers to a class of methods that use wavelets to estimatedensities or denoise data. The result of this transformation is anoverall reduction in variation with respect to the local mean of log₂ratios. In this context, “local” means a region consisting of a smallset of markers upstream and downstream from a given marker. Theresulting data can be viewed as the Weighted Log₂ Ratio in a genomebrowser program, such as the Chromosome Analysis Suite (Affymetrix,Inc., Santa Clara, Calif.) and serves as the input to the segmentationalgorithm. The wavelet shrinkage method is augmented by a reduction ofinfluence of outliers when local means are computed.

In the following examples, the log₂ ratios are transformed to a Harrwavelet basis and the wavelet coefficients are shrunk. A wavelet is afunction that integrates to zero and when squared has a boundedintegral. In order to reduce the influence of outliers, the differencebetween the shrunk values and the observed values are assumed to bedistributed student-t with 6.5 degrees of freedom. The log₂ ratios inputto the wavelet transform are weighted by the precisions obtained byfitting the residuals to a student-t distribution. Obtaining theprecisions requires iteratively computing the wavelet transform,obtaining residuals, then weights. When the vector of precisionsconverges the inverted transformation produces shrunk log₂ ratios andthen the HMM algorithm is employed to make copy number calls that areintegers. Here, the vector of copy number calls was segmented intocontiguous groups of like integers with a minimum segment size of 5.

The likelihood precisions of the shrinkage estimator were 0.5 for thefinest level of the transform and 4.0 for the next to finest. Theremaining levels were not shrunk. The precision of the Markov randomfield used for the prior was 0.5. The Markov random field employed inthe process is commonly referred to as a region quadtree. A quadtree iscommonly known as a tree data structure in which each internal node hasexactly four children. Quadtrees can be commonly employed to partition atwo-dimensional space (such as the surface of a microarray) bysubdividing the space into four quadrants, and then subdividing each ofthose four quadrants into four more quadrants, and repeating thisprocess over and over. This process decomposes the space into adaptablecells each having a maximum capacity. A region quadtree splits the spaceup multiple times into four equal quadrants of space. Each node in thetree then has exactly four children or no children. Each subdividedquadrant of space may have a depth of n and may represent an image of2^(n)×2^(n) pixels and each pixel may have a value of 0 or 1. If theregion does not have a value of 0 or 1, it may be again subdivided intofour more subquadrants. By subdividing the space in this way, thecalculations may be processed much more quickly and efficiently withoutexpending too much processing power to perform the necessarycalculations.

An exemplary summary of operations:

-   1. Set the precision log₂ ratios at all markers equal-   2. Iteratively until convergence    -   a. Compute the wavelet transform weighted by precisions    -   b. Shrink the wavelet coefficients    -   c. Invert the wavelet transform to obtain the shrunk log₂ ratios    -   d. Compute the residuals of the observed minus shrunk log₂        ratios    -   e. Update the precisions-   3. Call the copy number at each marker using the Cyto2 HMM-   4. Segment the copy number calls-   5. Smooth over any segments with fewer than 5 markers

In these analyses the variance of the shrunk log₂ ratios issubstantially less than their observed counterparts. The number of shortsegments is also substantially fewer using the restored signal. Longersegments are almost the same for both inputs with the major differencesbeing along stretches where the log₂ ratios are centered on a valueintermediate between two copy numbers.

Algorithmic Details of Wavelet Shrinkage

To execute wavelet shrinkage the log₂ ratios are transformed using aHarr wavelet bases which provides a multi-resolution representation ofthe data vector through recursive averaging as shown for a series of 8values x[1,1], . . . , x[1,8].

The first set of averages are:

-   -   x[2,1]=(x[1,1]+x[1,2])/2    -   x[2,2]=(x[1,3]+x[1,4])/2    -   x[2,3]=(x[1,5]+x[1,6])/2    -   x[2,4]=(x[1,7]+x[1,8])/2

The second set of averages are:

-   -   x[3,1]=(x[2,1]+x[2,2])/2    -   x[3,2]=(x[2,3]+x[2,4])/2

The third set of averages are:

-   -   x[4,1]=(x[3,1]+x[3,2])/2

Corresponding to averages are the first set of differences, which aredefined as:

-   -   y[2,1]=x[1,2]−x[1,1]    -   y[2,2]=x[1,4]−x[1,3]    -   y[2,3]=x[1,6]−x[1,5]    -   y[2,4]=x[1,8]−x[1,7]

The second set of differences may be as follows:

-   -   y[3,1]=x[2,2]−x[2,1]    -   y[3,2]=x[2,4]−x[2,3]

The third set of differences are:

-   -   y[4,1]=x[3,2]−x[3,1]

For any set of values x[1,1], . . . , x[1,N] where N is a power of 2,the recursion of averages and differences follows the exact samepattern. If N is not a power of 2 the vector of observations can bepadded with zeros until a number of elements with a power of 2 isreached. The original data can be recovered from the abovetransformation as follows:

-   -   x[1,1]=x[2,1]−y[2,1]/2    -   x[1,2]=x[2,1]+y[2,1]/2    -   x[1,3]=x[2,2]−y[2,2]/2    -   x[1,4]=x[2,2]+y[2,2]/2    -   x[1,5]=x[2,3]−y[2,3]/2    -   x[1,6]=x[2,3]+y[2,3]/2    -   x[1,7]=x[2,4]−y[2,4]/2    -   x[1,8]=x[2,4]+y[2,4]/2

The values x[2,1], . . . , x[2,4] used above can be recovered likewiseas:

-   -   x[2,1]=x[3,1]−y[3,1]/2    -   x[2,2]=x[3,1]+y[3,1]/2    -   x[2,3]=x[3,2]−y[3,2]/2    -   x[2,4]=x[3,2]+y[3,2]/2

Finally,

-   -   x[3,1]=x[4,1]−y[4,1]/2    -   x[3,2]=x[4,1]+y[4,1]/2

Wavelet shrinkage works on the y values which are often called thedetails. Let the details at the finest level of resolution be y[2,1], .. . , y[2j], . . . , y[2,P2] and let their shrunk values be z[2,1], . .. , z[2,j], . . . , z[2,P2]. Let q2 be a constant scalar value greaterthan zero. Let z[2j]=(q2(z[2 j−1]+z[2j+l])/2+y[2j])/(q2+1). In theequation above z[2j] is a weighted average of its adjacent neighbors andy[2j]. Implicit in this formula is that the z values are a priorisampled from a Markov random field. The detail y[2j] can be computeddirectly from the data, however, the sequence of values z[2,1], . . . ,z[2,p2] is unknown and must be solved iteratively. First a set of valuesfor z[2,1], . . . , z[2,p2] is proposed, then each z[2j] is computedaccording to what the current values of its neighbors.

The updating of the sequence is repeated until a convergence criterionhas been met. There are a number of sensible criterions to use. Thecurrent method is to stop when no z[2j] changes by more than 0.00001from one iteration to the next.

This process can be repeated for y[3,1], . . . , y[3,p3] using a weightq3 to generate a sequence z[3,1], . . . , z[3,p3] and so forth downthrough the recursive decomposition of the data. In practice it is onlynecessary to do this for the first two levels of the decomposition.

To reconstruct the shrunk signal replace the y values with the z valuesas follows using the example sequence of eight observations. Let theshrunk reconstructed signal be u[1,1], . . . , u[1,8].

-   -   u[1,1]=u[2,1]−z[2,1]/2    -   u[1,2]=u[2,1]+z[2,1]/2    -   u[1,3]=u[2,2]−z[2,2]/2    -   u[1,4]=u[2,2]+z[2,2]/2    -   u[1,5]=u[2,3]−z[2,3]/2    -   u[1,6]=u[2,3]+z[2,3]/2    -   u[1,7]=u[2,4]−z[2,4]/2    -   u[1,8]−u[2,4]+z[2,4]/2

The values u[2,1], . . . , u[2,4] used above can be recovered likewiseas:

-   -   u[2,1]=u[3,1]−z[3,1]/2    -   u[2,2]=u[3,1]+z[3,1]/2    -   u[2,3]=u[3,2]−z[3,2]/2    -   u[2,4]=u[3,2]+z[3,2]/2

Finally,

-   -   u[3,1]=x[4,1]−y[4,1]/2    -   u[3,2]=x[4,1]+y[4,1]/2

The new shrunk values u[1,1], . . . , u[1,N] are used in the hiddenMarkov model. Samples with good quality control metrics produce almostidentical patterns of segments with 25 markers or more.

Only one level of resolution is required for describing how outliers arehandled so write 15 the sequence of observed values x[1], . . . , x[N]and do the same for corresponding sequences. In the sequence of valuesx[l], . . . , x[N], large outliers can be present. The outliers areaddressed by iteratively downweighting their influence on the finalsolution of u[1], . . . , u[N] as follows. First u[1], . . . , u[N] arecomputed from x[1], . . . , x[N] using the wavelet shrinkage method. Avector of errors e[1], . . . , e[N] is computed by taking the differencebetween the x and u values. Large 20 outliers produce large values ine[l], . . . , e[N]. A weight can be attributed to each error by thefollowing formula w[j]=[(k+1)/(kS+(x[j]−u[j])(x[j]−u[j]))]/wmean.

The variable k is called the degrees of freedom and a value of 6.5 isused in the current implementation. The variable wmean is the mean ofall w[j] had then not been divided by wmean. S is a scaling factor setsuch that the sum (x[1]−u[1])(x[1]−u[1])+ . . .+(x[N]−u[N])(x[N]−u[N])=NS.

Implicit in the above calculations is that the members of the errorvector when divided by the square root of S are distributed student-twith 6.5 degrees of freedom. Next a set of values v[1], . . . , v[N] arecomputed using the formula v[j]=u[j]+w[j]e[j]. The values v[l], . . . ,v[N] are now shrunk instead of the values x[l], . . . , x[N] and a newset of errors are computed where e[j]-x[j]−u[j]. Notice the outlier isstill computed from the observed value.

This process is ideally repeated until the vector u[l], . . . , u[N]converges but instead the iterations are truncated. The currentimplementation terminates after 15 iterations by default.

Copy Number High Pass Filter

Many microarrays comprising DNA probes mounted thereon exhibit artifactswhich arise during the manufacturing process or after binding of targetsample, as discussed above. For instance, microarrays manufactured usinga photolithographic process in which single monomers are added to thesurface one step at a time require the use of masks in order to shieldsome areas of the array surface while other areas undergo chemicalreaction. Sometimes the manner in which the mask is used or manufacturedcan introduce various artifacts in how the image of the log₂ ratio ofthe signal intensity data is generated and displayed. In some instances,there may be horizontal bands which may be apparent to the human eyeupon visual inspection. These bands may be somewhat brighter or darker,each band possessing its own hue or degrees of different shades ofcolor. If a microarray is positioned on a flat surface in front of theuser, there may be designated theron a top, bottom, left side and aright side to the array when looking down at the array from a vantagepoint perpendicular to the plane of the array. The horizontal bands may,for instance, be somewhat brighter in shade on the left side of thearray than on the right side, or vice versa. The bands may have brighterand darker regions from left to right, independently of other bandsabove or below it in the array. These types of shading artifacts may becaused inadvertently as a byproduct of the inherent differences in theaffinity of various probes on the array for their target and how thoseprobes may be placed, the sequence of the probes, etc. as described indetail with respect to the various covariate adjusters used to minimizethese differences.

As already mentioned, above, the present invention may employ solidsubstrates for microarrays. Methods and techniques applicable to polymer(including protein) array synthesis have been described in U.S. Ser. No.09/536,841 (abandoned), WO 00/58516, U.S. Pat. Nos. 5,143,854,5,242,974, 5,252,743, 5,324,633, 5,384,261, 5,405,783, 5,424,186,5,451,683, 5,482,867, 5,491,074, 5,527,681, 5,550,215, 5,571,639,5,578,832, 5,593,839, 5,599,695, 5,624,711, 5,631,734, 5,795,716,5,831,070, 5,837,832, 5,856,101, 5,858,659, 5,936,324, 5,968,740,5,974,164, 5,981,185, 5,981,956, 6,025,601, 6,033,860, 6,040,193,6,090,555, 6,136,269, 6,269,846 and 6,428,752, and in PCT ApplicationsNos. PCT/US99/00730 (International Publication No. WO 99/36760) andPCT/US01/04285 (International Publication No. WO 01/58593), which areall incorporated herein by reference in their entirety for all purposes.Patents that describe synthesis techniques in specific embodimentsinclude U.S. Pat. Nos. 5,412,087, 6,147,205, 6,262,216, 6,310,189,5,889,165, and 5,959,098. Nucleic acid arrays are described in many ofthe above patents, but the same techniques are applied to polypeptidearrays and may be applied to other types of arrays utilizing variousmonomers.

Thus, when using a computer-generated visual image of such microarrays,the problem of variation in shading on the surface of the array fromfeature to feature may be detected and removed from the experiment in anumber of ways, just as the use of covariate adjusters also are able tonormalize various inherent properties and performance of the probes andtarget in a microarray experiment. One way to remove these artifactsfrom the microarray experiment is to filter the array data to removethese artifacts using software and algorithmic methods. In a generalsense, the present methods include constructing a theoretical2-dimensional plane having no variation in signal intensity, and thenlooking for intensity values on the array which match the theoreticalfield. Once such a position or spot is found on the microarray whichmatches a theoretical plane having no intensity or color variation, theremaining areas of the microarray which do not conform to thistheoretical field may be adjusted to be either darker or lighter inshade such that their underlying intensity values are equivalent, orroughly equivalent, to the theoretical field values. These adjustmentsmay then be applied to the experimental data as well, as a negativecontrol removing background noise from the data.

Much research and thought has already been published in statisticalmethods which may be applied to such problems of color normalization.For instance, see “On the Statistical Analysis of Dirty Pictures,” byJulian Besag, J. R. Statist. Soc. B, 48(3):259-302, 1986, incorporatedherein by reference in its entirety for all purposes. This publicationdiscloses the partitioning of a two-dimensional region into finerectangular arrays of sites or “pixels” wherein it is assumed that“pixels close together tend to have the same or similar colours.” (Id.)

The process of locating on a planar two-dimensional field a region ofsimilar color or intensity and then adjusting all other areas in thefield to the same levels as the found region can be referred to as atype of “signal restoration.” What one is doing is in fact “restoring”to “normal” the artificially highlighted or depressed colors or signalintensities of various regions of the field (the array surface) so thatthe entire field is the same, uniform, “normal” color or intensity.Thus, the “normal” signal values, which would have appeared had themanufacturing process been perfect and without variation or flaws, maybe restored to the microarray being examined in the experiment.

The presently disclosed data filter methodology applies signalprocessing techniques to manipulation of image data. In embodiments ofthe present methodologies, the image manipulated is actually apseudo-image based on log₂ ratio values, not raw images based on rawdata obtained from standard .DAT or .CEL files. Transforming the rawdata to log₂ ratio space allows visualization and detection of importantsignal gradients that can only be seen with this transformation andthereafter corrected.

The model used to achieve this filtering is the following:

y _(i) =z _(i) +x _(i)

where y_(i) is the computed log₂ ratio, z_(i) is the true unobservedlog₂ ratio, and x_(i) is the bias of observed values due to nuisancelow-frequency information in the image. This bias is itself a randomvariable with the expected value of x_(i) being that of its immediatelysurrounding region.

Motivation for the model was driven by the observation that residualsfrom copy number analysis were biased by a gradual spatial trend. Whencopy number data are compared to a reference data set, the expectationis that in most cases the logs ratio is zero. This was clearly not thecase. A secondary benefit of this methodology is the decreasedconsumption of RAM, or computer memory. Consumption of RAM is reduced byanalyzing the bands of copy number probes.

In any band of copy number probes there are missing values. On a copynumber chip the locations of missing values are always known ahead oftime.

Spatial Distribution of Signal

A key assumption in employing the present methodology for high passfiltering of image data is that there is no long range spatialcorrelation of the true value of the log 2 ratio-corrected copy numbersignal, z. This feature only exists in y through x. This assumption,together with the symmetric zero-mean distribution of z, anchors theestimate of the background through the data.

For almost all of the genome z will have an expected value of zero.Because of the pseudo-random spatial allocation of copy number probes,the observed values can be modeled as random. The log transformationrenders the log₂ ratios unimodal and nearly symmetric. They are modeledwell as zero-mean normal deviates with variance of σ².

Local Estimate of the Background

The background can be modeled without data. Any background element x_(i)is distributed normal with expected values being a weighted average ofsurrounding probes. The weights are just precisions. Adjacent probes atthe sides with a common edge have a precision of τ. Adjacent probes inthe corners have a precision of:

$\frac{\tau}{\sqrt{2}}$

The larger the value of τ, the more smooth is the background. Withoutdata, the level of the background is arbitrary. With data, the level ofthe background becomes anchored.

Estimating Dispersion Parameters

The background forms a type of prior distribution, called a MarkovRandom Field. One feature of this model is that the likelihood and priorcan be written explicitly. Also, all the full conditional distributionsare available for computational purposes. It turns out that estimating 7is difficult. Since we are not actually modeling the data, but ratherusing a model for trend compensation, the product of τσ² is insteadfixed. A default value of τσ²=8 produces a smooth background.

Imposing a Hierarchy

Enforcing smooth long term trends with the model above would require theproduct a&² to be a very large value. Convergence would move at aglacial place. In order to overcome this, a multi-grid approach isemployed where each x_(i) is located in one corner of a block of four.These blocks form a blurred version of the original data at one quarterthe size. The observed values are averaged. This can be done recursivelyuntil, at the highest level, no more blocks of four exist. The expectedvalue fir the background then becomes an average of its neighbors, andthe background of the two-by-two block in which it resides. Thetwo-by-two block uses the same smoothing parameters. This is recursive.Increasing the weight of the two-by-two block increases the effect ofthe hierarchy and likewise the global effect of smoothing.

The hierarchy makes convergence quite rapid.

Blocking the Data for Speed

A modification to make the application faster is to aggregate theobserved values into blocks. Essentially, start higher up in thehierarchy under the assumption that the background is basically the samefor small blocks. Currently 8×8 blocks are used to speed execution time.This is not to be confused with slow kernel smoothing methods thatrequire blocks to be large in order to obtain long range smoothness witha smaller sized kernel. A 512 by 1024 image with gaussian noise and nomissing values is used to evaluate the estimate of the background.Various trends are added to the image to evaluate results.

-   -   At the lowest level of the hierarchy set all x_(i) to 0.    -   Divide the image into blocks of 4 and average the z_(i) to form        the next level.    -   Recursively repeat the first two steps at the next level in        hierarchy until no more blocks of 4 can be formed.    -   Using an iterative method, solve for the background at the top        level.    -   Cascade down the hierarchy solving for the background each time    -   At the bottom level the z, can now be read off.

It is to be understood that the above description is intended to beillustrative and not restrictive. Many variations of the invention willbe apparent to those of skill in the art upon reviewing the abovedescription. The scope of the invention should be determined withreference to the appended claims, along with the full scope ofequivalents to which such claims are entitled. All cited references,including patent and non-patent literature, are incorporated herewith byreference in their entireties for all purposes.

1.-10. (canceled)
 11. A computer-implemented method of correcting a dataset comprising one or more intensity values from a microarrayexperiment, the method comprising: scanning, by a scanning device, oneor more nucleic acid samples hybridized to a nucleic acid probe mountedon a microarray used for the microarray experiment such that a data setcomprising one or more intensity values from the microarray experimentis obtained; detecting, by a processor, one or more artifacts in thedata set by calculating logarithmic ratio values of at least some of theone or more intensity values based on one or more reference valuesstored in a memory, and using the logarithmic ratio values to detectsignal gradients in the data set; applying, by the processor, at least afilter to the logarithmic ratio values to remove the one or moreartifacts, thereby generating a filtered data set; applying, by theprocessor, one or more algorithms to the filtered data set to normalizethe filtered data set at least with respect to one of intensity orcolor, thereby generating a corrected data set; and displaying at leasta portion of the corrected data set as a graphical presentation withinan electronic user interface.
 12. The method of claim 11, wherein theone or more artifacts are caused by a variation in shading on a surfaceof the microarray.
 13. The method of claim 12, wherein the variation inshading on the surface of the microarray is an artificially highlightedcolor in at least one region of the surface of the microarray.
 14. Themethod of claim 12, wherein the variation in shading on the surface ofthe microarray is an artificially depressed color in at least one regionof the surface of the microarray.
 15. The method of claim 11, wherein atleast one of the one or more reference values corresponds to atheoretical plane having no signal intensity or color variation.
 16. Themethod of claim 15, wherein the theoretical plane is a two-dimensionalplane.
 17. The method of claim 11, wherein the logarithmic ratio valuesare log 2 ratios.
 18. The method of claim 11, wherein the logarithmicratio values are centered at a value of zero.
 19. The method of claim11, wherein the one or more algorithms are applied to the filtered dataset to remove a spatial bias.
 20. The method of claim 11, wherein theone or more algorithms are applied to the filtered data set to lower alevel of background noise in the filtered data set.
 21. The methodaccording to claim 11, wherein the one or more algorithms comprise Bayeswavelet shrinkage to reduce variations in the logarithmic ratio values.22. The method according to claim 21, further comprising: transforming,by the processor, the logarithmic ratio values using a Haar waveletbasis.
 23. The method of claim 11, wherein the microarray comprises asolid substrate.
 24. The method according to claim 11, wherein themicroarray experiment comprises an expression level determinationexperiment.
 25. The method according to claim 11, wherein the microarrayexperiment comprises a copy number determination experiment.
 26. Themethod of claim 11, further comprising determining a copy number basedon the corrected data set.
 27. The method according to claim 11, whereinthe data set comprises one or more intensity values obtained from amicroarray experiment conducted on a human DNA sample.
 28. The methodaccording to claim 11, wherein the one or more nucleic acid samplescomprise a labeled human DNA sample and the one or more intensity valuesare obtained by scanning fluorescence intensities of the labeled humanDNA sample hybridized to nucleic acid probes on the microarray.
 29. Themethod according to claim 11, wherein the graphical presentationindicates at least one of expression levels, genotypes, copy numbers,and loss of heterozygosity (LOH) for the microarray experiment based onthe corrected data set.